=~ Avda Valalersli eas) aie) apatclale= 
Report and FY 2019 Annual 
eXelacelaaitslalecm aatsla 


U.S. Department of Education 
Betsy DeVos 
Secretary 


February 12, 2018 


This report is in the public domain. Authorization to reproduce it in whole or in part is granted. 
While permission to reprint this publication is not necessary, the citation should be: U.S. 
Department of Education, Fiscal Year 2017 Annual Performance Report and Fiscal Year 2019 
Annual Performance Plan: Washington, D.C., 2018. 


Notice of Availability of Alternate Formats 


This report is available at http://www.ed.gov/about/reports/annual/index.html. On request, 
the report also is available in alternative formats, such as Braille, large print, or compact disk. 


For more information, please contact our Alternate Format Center at 202-260-0852 or the 504 


coordinator via e-mail at om_eeos@ed.gov. 


To become connected to the Department through social media, please visit the Department’s 
website at www.ed.gov. Our Twitter page is at @usedgov, and our blog is at Homeroom. 


Notice to Limited English Proficient Persons 


Notice of Language Assistance: If you have difficulty understanding English, you may request 
language assistance services, free of charge, for this Department information by calling 1-800- 
USA-LEARN (1-800-872-5327); (TTY: 1-800-877-8339), or by e-mailing us at 
Ed.Language.Assistance@ed.gov. 


Please submit your comments and questions regarding this report, and any suggestions to 


improve its usefulness to APP_APRComments@ed.gov or write to: 


U.S. Department of Education 
Performance Improvement Officer 
400 Maryland Ave, SW 
Washington, DC 20202 


Table of Contents 


WaADle OF COMPING wiv vesscn cscs dcaneteideasasnesionaceatancaninnasencnduansdcnnedeniteametesnenmuaiwadinencsaninrkaaisentendetdennnnens 3 
AbDOUE TNE De PartiME NE iien: ciceecsteesscesenssisasvensesenesessecsseos sees seven sedsseussenessensanns ss cuandcusicessdedsussdsssedeenes 4 
Operating: StrUClUne icc tesecsecettcesccscecccswedseeeleventecsenesswessdescidencesneucaceedsceedesssceseeedecescvensdusmmshstestesende 5 
Sg Se ee 6 
EY ZOOL? POPOrini Gia  stcicccsicestisasescsenadecnsacevenisniuaxsasieensernstdasesnanweceriiosiseutasesndanedeteedsdumusiestoacerninanes 6 
FY 2018-22 Strategic Goals and Strategic Objectives .........ccccccceesssesseeeeceeeeeeseeseeeeneteeeeenees 13 
The Department’s Agency Priority Goals (APGS)........:ccccccessssseeeeneeeeeeeeeeeeeeeseeceeeeeeeeeneeeeees 14 
Cross-Agency Priority (CAP) Goals ........::c:ccccesesseeeneeesneesennennneeneeneneeneasenaeeeaeenaneeasneessensneesnes 15 
Strategic GOal! Vere ticcse: ceeececes sseccccceeweseeesce tebe este ce edewcteee cet decuededucoescaeeweaddscavenseseesousuessssnaseesioanes 16 
(50a sl DISCIGliONaly RESOUlCSS si icccccn Ash cstacacceastentasteueaedisonts tancivagadsscunaranceenpachapacecetencantacannsadse 26 
Major Discretionary Programs and Activities Supporting Goal 1.0.0.0... eeeeeeeeeeeeeeteeeeeeeeees 26 
StrateGic GOA 2: sissies issescaesscesscciescacneseveenececsesvuucncusenceseccechesdseanvescadeededecsedveseteessadtruechesereedesdessersis 27 
(G0al 2 DISCIGUONALY RESOUICSS visscnictecnctatnaientat neni dneaehosedietnidnanscuadssieaecdeben tian ilaneidececcomebtanedins 37 
Major Discretionary Programs and Activities Supporting Goal 2.0.0.0... cece ceeeeeeeeeeeeeeeteeteeeeeeees 37 
ite h0| (a Ciel: |: 2 39 
(Oal:3 DISCKELONAlY RESOUICSS sooo ccnccesmnssantodesentddakandacuissiataunpensnpucaudveattaroraieiahacceendatldiiaises 44 
Major Discretionary Programs and Activities Supporting Goal 3.0.0.0... eeeeeeeeeeeeeeeeeeeeeeeeeeeees 44 
Strate GIS: GOA A ici. cvisescsesacenuezivucncudanceeevcesnecsshendhsescevsaseuscsdveensesedsesued¢uvedsbesterssesssescteeaseeesrieaeesei> 45 
(sal. 4 DISCFEllONaly RESOUICES xissicctscecascnsssancctioitingzaczanaysortedintstanaupuont hiaeisaantsageenoecaenaneneaate 51 
Major Discretionary Programs and Activities Supporting Goal 4.0.0.0... eee eeeeeeeeeeeeeeeeeeeteeeeeeeees 51 
Major Management Priorities and Challenges .......cccccccseseseeeeeeeeeeeeeeeeeeeneeseeeeeenseeseeeeeeeeeeenees 53 
PADD GNIGIGES sci cnacs ca svics svc avexcxedeessncatsccsuccussasusdsuzndechetussscdauecttuncsanneshissus sents tevansdeeeies feineecstenesaieeettee 57 
Appendix A: FY 2017 Performance Metrics ..............c:cccccseeceeeceeeeeeeeeeeeeaeeeaeeeaeeeaaeeaaesaaeeaeseaeaeea 58 
Appendix B: Data Validity and Verification for FY 2017 Annual Performance Repott............. 74 
Appendix C: Data Validity and Verification for FY 2019 Performance Metrics............. cee 93 
Appendbe D: PROOFaMS DY COP aca peseveccdevnsts ontitpeneec detache duce ttcantanleianeshcencraeeathandimnaadoeenseutallh 113 
Appendix E: Summary of Performance Evaluations Conducted During FY 2017 and Expected 
UTM Vk OE aos vtec tc sed eRccay dda cca caiietcie ets ae laremel pate attac ected toast Sieoecadeuntuidst 117 
Appendix F: Glossary of Acronyms and Abbreviations .0.0..........ececeeeeeeeeeeeeeeeeeeeenteeeeeeeeeeeeeeee 161 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 3 


DEPARTMENT OF EDUCATION MISSION 


About the Department 


The U.S. Department of Education’s mission is to promote student achievement and preparation 
for global competitiveness by fostering educational excellence and ensuring equal access. 


The Department accomplishes its mission by administering programs that provide services 
throughout an individual’s entire lifespan—from early intervention services to employment 
programs. Many of these programs provide grants to state educational agencies (SEAs) or local 
educational agencies (LEAs) and support students and families from vulnerable populations, 
including children with disabilities and those from disadvantaged backgrounds. These programs 
also provide grants and loans to postsecondary students, and facilitate research that examines 
ways that states, schools, districts, and postsecondary institutions can improve America’s 
education system. In addition, the Department fulfills its mission through the enforcement for 
civil rights laws that provide equal access to Department programs for all individuals. 


The figure below reflects the operating structure of the U.S. Department of Education used in 


Fiscal Year (FY) 2017. Interactive and text versions of the operating structure of the Department 
are available online. 
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Operating Structure? 
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1 Source https:/Awww2.ed.gov/about/offices/or/index.html. 
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Overview 


The FY 2017 Annual Performance Report (APR) and FY 2019 Annual Performance Plan (APP), 
required by the Government Performance and Results Act Modernization Act of 2010, 
accompanies the Department’s Budget submission and links performance goals with resources 
for achieving targeted levels of performance. The APR reflects performance against the 
Department’s FY 2014-18 Strategic Plan. The APP identifies implementation strategies and 
performance measures to accomplish the strategic goals and objectives represented in the 
Department’s FY 2018-22 Strategic Plan. 


The U.S. Department of Education’s mission is to promote student achievement and preparation 
for global competitiveness by fostering educational excellence and ensuring equal access. This 
mission is manifested in the Department’s efforts to continually improve the educational 
environment for all students, addressing their educational needs. The Department’s National 
Center for Education Statistics (NCES) estimates that 50.7 million students were attending 
public elementary and secondary schools in 2017, with a projected 35.6 million in 
prekindergarten through grade 8 and a projected 15.1 million in grades 9 through 12. An 
additional 5.2 million students are projected to attend private elementary and secondary 
schools. NCES predicts that the total pre-K—12 enrollment will continue to grow to an all-time 
high of 56.8 million by 2026, indicating the increasing need for quality education throughout the 
United States. 


FY 2017 Performance 


Education is primarily a state and local responsibility in the United States. It is states and 
communities, as well as public and private organizations of all kinds, that establish schools and 
colleges, develop curricula, and determine requirements for enrollment and graduation. As 
such, the Department is committed to supporting SEA, LEAs and other education institutions to 
help ensure every student in America has an equal opportunity for a great education. 


In March 2017, the Department released a revised consolidated state plan template to support 
states in meeting the requirements of the Elementary and Secondary Education Act of 1965 
(ESEA), as amended by the Every Student Succeeds Act (ESSA). The Department worked with 
SEAs, and other state and local stakeholders, to develop a revised template that was structured 
to reduce burden and promote innovation, flexibility, transparency, and accountability, while 
maintaining essential protections for all students. The revised template asked states only to 
provide details on their plans in areas (a) explicitly required by law and (b) deemed absolutely 
necessary for consideration of such a plan, consistent with ESEA section 8302(b)(3), leveraging 
input of states, local educators, and parents. State plans were submitted to the Department and 
peer-reviewed. 


In FY 2017, the Department also focused on promoting evidence-based decision making with 
the intention to support states and districts in using and building evidence effectively. The 
Department published revised evidence definitions and related selection criteria for competitive 
grant programs in the Education Department General Administrative Regulations that align with 
ESSA, disseminated non-regulatory guidance on evidence in ESSA, Using Evidence to 
Strengthen Education Investments; awarded $16 million to support rigorous evaluations and 
researcher-practitioner partnerships focused on state and local education priorities. 


With regard to postsecondary education, the passage of the FY 2017 spending bill restored 
year-round Pell grants, and the Department announced that these grants would become 
available to college students beginning July 1, 2017. The Department recommended that unless 
a student had remaining eligibility from the 2016-17 award year, institutions should award Pell 
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Grant funds for this past summer out of the 2017-18 award year since the additional funding will 
be available later in the year (e.g., spring or summer of 2018). The change allows an eligible 
student to receive up to 150 percent of the student’s scheduled Pell Grant for an award year 
beginning with the 2017-18 award year, giving hundreds of thousands of students more 
resources to finish their coursework in a timeframe that meets their individual needs. 


In FY 2017, the Department also paused on two postsecondary regulations—Borrower Defense 
to Repayment, concerning forgiveness of student loan debt, and Gainful Employment, 
concerning educational programs that prepare students for gainful employment in a recognized 
occupation. Two negotiated rulemaking committees have been established to rethink these two 
higher education regulations, with the intent to develop fair, effective, and improved regulations 
to protect individual borrowers from fraud, ensure accountability across Institutions of Higher 
Education (IHEs), and protect taxpayer interests. It is the Department’s aim to protect students 
from predatory practices while also providing clear, fair, and balanced rules for colleges and 
universities to follow. 


The Department continued to improve the information and data it makes available to parents, 
students, and education stakeholders. A few examples include: 


e The Department's College Scorecard, which supports postsecondary students by 
providing the public with clear, easily accessible information on college performance. 

e The Department's National Assessment of Educational Progress (NAEP), the largest 
nationally representative and continuing assessment of student knowledge in various 
subject areas. 

e The Department's InformED initiative, which is intended to transform how the 
Department makes information available—and actionable—for internal users and for the 
public through the identification and development of open data initiatives. 


Finally, the Department continued to strengthen its internal operations throughout FY 2017. In 
response to President Trump’s Executive Order 13777, Enforcing the Regulatory Reform 
Agenda, the Department established a Regulatory Reform Task Force that catalogued over 150 
regulations and more than 1,700 items of policy guidance at the Department. The task force, 
comprised of agency political appointees and career staff, provided recommendations on which 
regulations and guidance documents to repeal, modify, or keep in an effort to ensure those 
items that remain protect students while giving states, institutions, teachers, parents, and 
students the flexibility to improve student achievement. 


Also, in response to Executive Order 13781, Comprehensive Plan for Reorganizing the 
Executive Branch, the Department submitted an Agency Reform Plan to the Office of 
Management and Budget (OMB). Department staff continue to assess reform factors that 
include: new activities or functions the Department should initiate; ways the agency can be more 
efficient in meeting the needs of students, families, and education partners; activities or 
functions the Department should consider combining or modifying; agency activities or functions 
that duplicate what others are doing; and how the Department could best deliver education 
services or products to students and educators. 


See Appendix A for detailed information on the FY 2017 performance measures. 


The Department uses quarterly performance reviews, targeted strategic initiatives, and outreach 
to leaders and stakeholders to assess progress and garner engagement toward achieving 
strategic goals and outcomes. Looking to the future towards the FY 2018—22 Strategic Plan, the 
Department plans to focus in four key areas: (1) supporting state and local efforts to improve 
learning outcomes for all P—12 students in every community; (2) expanding postsecondary 
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education opportunities and improving outcomes to foster economic opportunity and informed, 
thoughtful and productive citizenry; (3) strengthening the quality, accessibility and use of 
education data through better management, increased privacy protections, and transparency; 
and (4) reforming the effectiveness, efficiency and accountability of the Department. 


Through its FY 2018-22 Strategic Plan, the Department is focusing on the students it serves. 
Students have unique needs, and the Department must strive to align and tailor its programs to 
meet their needs. When thinking about how education delivery and the support the Department 
provides to parents, teachers, school leadership, districts, institutions, and states must change, 
it is important to reflect on the current state of education in the United States. The following 
graphics provide information on public school funding, state performance compared to the 
Nation on the NAEP, high school graduation rate and degree attainment rate. 


Public Funding Sources 


Distribution of public elementary-secondary 
THC ES TSC CSC Se Cr Le 


91.7% Local/State; 8.3% Federal 


1.2% Other Local Governments 


Local 44.6% State 47.1% ‘L___ Federal 
8.3% 


Source: U.S. Census Bureau, 2015 Annual Survey of School System Finances, Table 5,"Percentage Distribution of Public Elementary-Secondary School 
System Revenue by Source and State: Fiscal Year 2015” <https://www.census.gov/content/dam/Census/library/publications/2017/econ/g15-aspef.pdf 
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National Assessment of Educational Progress — Fourth Grade 
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State Performance Compared to the Nation 
Reading, Grade 4 
Difference in average scale scores between all jurisdictions and National public, for All students 
[TOTAL], 2013 
ame 


221 


4 
in average scale scores between all jurisdictions and National public, for All students [TOTAL] , 2013 


+ Not applicable. 

# Rounds to zero. 

NOTE: DS = Department of Defense Education Activity (DoDEA). The NAEP Reading scale ranges from 0 to 
500. Some apparent differences between estimates may not be statistically significant. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment. 


State Performance Compared to the Nation 

Mathematics, Grade 4 

Difference in average scale scores between all jurisdictions and National public, for All students 

[TOTAL], 2015 
nated 


240 


Dafference in dverage scale scores between all jerisdictions and National public, for All studeats [TOTAL] ,2015 


+ Not applicable. 

# Rounds to zero. 

NOTE: DS = Department of Defense Education Activity (DoDEA). The NAEP Mathematics scale ranges from 
0 to 500. Some apparent differences between estimates may not be statistically significant. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment. 
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National Assessment of Educational Progress — Eighth Grade 


State Performance Compared to the Nation 
Reading, Grade 8 
Difference in average scale scores between all jurisdictions and National public, for All students 
[TOTAL], 2013 
2013 National public (NP) average 
‘score (0-500) 


[266 


a orm 


Reading, grade 8 
Difference in average scale scores between all jurisdictions and National public, for All students [TOTAL] , 2013 


+ Not applicable. 

# Rounds to zero. 

NOTE: DS = Department of Defense Education Activity (DoDEA). The NAEP Reading scale ranges from 0 to 
500. Some apparent differences between estimates may not be statistically significant. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment. 


State Performance Compared to the Nation 
Mathematics, Grade 8 


Difference in average scale scores between all jurisdictions and National public, for All students 
[TOTAL], 2015 


2015 National 
word oo aa 


[281 


+ Not applicable. 

# Rounds to zero. 

NOTE: DS = Department of Defense Education Activity (DoDEA). The NAEP Mathematics scale ranges from 
0 to 500. Some apparent differences between estimates may not be statistically significant. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment. 
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National Assessment of Educational Progress — Twelfth Grade 


State Performance Compared to the Nation 

Reading, Grade 12 

Difference in average scale scores between all jurisdictions and National public, for All students 
[TOTAL], 2013 


2013 National public (NP) average 39 
‘score (0-500) 
junacictions 
Reading, 12 
Dimers average scale scores between all jurisdictions and Nationsl public, for All students [TOTAL] ,2013 
National 
public 
Cg 
Ri 
si oe? 
DE 
MO 
oc 


— Not available. 

+ Not applicable. 

NOTE: DS = Department of Defense Education Activity (DoDEA). The NAEP Reading scale ranges from 0 to 
500. Some apparent differences between estimates may not be statistically significant. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment. 


State Performance Compared to the Nation 
Mathematics, Grade 12 
Difference in average scale scores between all jurisdictions and National public, for All students 
[TOTAL], 2013 
einen lees 


152 


Mathematics, 
Difference in average scale scores between all jurisdictions and National public, for All students [TOTAL] , 2013 


Not available. 
+ Not applicable. 
NOTE: DS = Department of Defense Education Activity (DoDEA). The NAEP Mathematics scale ranges from 
0 to 300. Some apparent differences between estimates may not be statistically significant. 
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment. 
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National High School Graduation Rate 


C . 


National College Degree Attainment 


College Degree Attainment for persons age 25-34 


> Degree Attainment 
by Gender and Race/Ethnicity 


Degree Attainment Degree Attainment by 
by Gender Race / Ethnicity 
52% 
—— Male Female White Black ere Asian Other 
Oe ae Degree Type 


-> 53.5% 


Associate's degree Bachelor'sdegree = Master's degree First-professional 
or doctor's degree 


Source: U.S, Department of Commerce, Census Bureau, Current Population Survey (CPS), Annual Social and Economic Supplement, 2016. (This table was 
prepared March 2017.) <https://nces.ed.gov/programs/digest/d16/tables/dt16_104.30.asp?current=yes>. 
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FY 2018-22 Strategic Goals and Strategic Objectives 


Strategic Goal 1: Support state and local efforts to improve learning outcomes for all P-12 students in every 


community. 


Increase high-quality educational options and empower students and parents to 


Sr ee caveat choose an education that meets their needs. 


Strategic Objective 1.2 Provide all P-12 students with equal access to high-quality educational opportunities. 


Prepare all students for successful transitions to college and careers by supporting 
Strategic Objective 1.3 access to dual enrollment, job skills development and high-quality science, technology, 
engineering and mathematics (STEM). 


Support agencies and institutions in the implementation of evidence-based strategies 
Strategic Objective 1.4 and practices that build the capacity of school staff and families to support students’ 
academic performance. 


Strategic Goal 2: Expand postsecondary educational opportunities, improve outcomes to foster economic 
(o) 0) ofe)atvlalinvar-lalem e)nelaate) (ome lam alco) anat-remmaaelo(e) alae) mr- laren o)celelU(eli\(omelliy4>19]aVm 


Strategic Goal 3: Strengthen the quality, accessibility and use of education data through better management, 
increased privacy protections and transparency. 


Improve the Department’s data governance, data life cycle management and the 


SUES eo CU Set oat capacity to support education data. 


Improve privacy protections for, and transparency of, education data both at the 


Seo Te Seu ie Department and in the education community. 


Increase access to, and use of, education data to make informed decisions both at the 


Strategie Objective 2:3 Department and in the education community. 


Strategic Goal 4: Reform the effectiveness, efficiency and accountability of the Department. 


Provide regulatory relief to educational institutions and reduce burden by identifying time- 
Strategic Objective 4.1 consuming regulations, processes and policies and working to improve or eliminate them, 
while continuing to protect taxpayers from waste and abuse. 


Strategic Objective 4.2 Identify, assess, monitor and manage enterprise risks. 


Strengthen the Department’s cybersecurity by enhancing protections for its information 


eee ToT laos: technology infrastructure, systems and data. 


Improve the engagement and preparation of the Department’s workforce using 


SHEMSS ECU SS Ie professional development and accountability measures. 
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The Department’s Agency Priority Goals (APGs) 


The Department identified four Agency Priority Goals (APGs) for FY 2018-19. Improving 
education starts with allowing greater decision-making authority at the state and local levels and 
empowering parents and students with educational options. These APGs aim to increase 
educational choice, improve the customer service the Department provides, ensure students are 
protected, and reduce red tape. The effective implementation of the Department's APGs will 
depend, in part, on the effective use of high-quality and timely data, including evaluations and 
performance measures. Quarterly updates for the APGs will be available on 


www.Performance.gov. 


APG Related Strategic Objective 


Improve the access to, and the quality and transparency of, Strategic Objective 1.1: Increase high- 
Tod alole) Med ate)(e- Me) oli(o) alm ie) men AJ aUle(-1 aim quality educational options and empower 
students and parents to choose an 

By September 30, 2019, the Charter School Program (CSP) will education that meets their needs. 
10) 0) ole) amiat=meig=r-li(e)aur-lalem=>.¢0r-lalcl(e) ae) mci0 10M aloW mei ate lalz) meses alee) (s 

nationally. The CSP will also support the enrollment of 50,000 students 

in new charter schools. Additionally, by September 30, 2019, the 

Department will disseminate eight resources, at least one per quarter, 

on evidence-based and promising practices related to school choice. 


Improve borrowers’ access to quality customer service. 


By September 30, 2019, Federal Student Aid will improve customers’ 
access to and availability of quality customer service by decreasing the 
overall average speed of answer to 60 seconds or less, decreasing 
abandoned rates to three percent or less and requiring all non-default 
federal student loan servicers to expand and standardize call center 
hours. 


Improve student privacy and data security at Institutions of Strategic Objective 3.2: Improve privacy 
altel atime Le cer-ud(elam (nl =-)Manlcelelelimeliic--Cum-lemeeeliclle-moiesce™ protections for, and transparency of, 
education data both at the Department and 
By September 30, 2019, the Department will increase information in the education community. 

security program outreach activities to IHEs by 40% in order to help 

protect IT systems and data privacy and commence audits of IHEs 

subject to A-133 and Gramm-Leach-Bliley Act (GLBA), resulting in 36 

IHEs (from a baseline of zero) completing an audit of GLBA-related 

information security safeguards with no significant findings. 


Provide regulatory relief to education stakeholders. Strategic Objective 4.1: Provide 
regulatory relief to educational institutions 

By September 30, 2019, the Department will reduce the regulatory and reduce burden by identifying time- 

burden on education stakeholders by submitting to OMB no less than consuming regulations, processes and 

25 deregulatory actions (against a baseline of zero (0) for FYs 2015 policies and working to improve or 

Flare WyA0 Ke) B eliminate them, while continuing to protect 


taxpayers from waste and abuse. 
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Cross-Agency Priority (CAP) Goals 


In addition to the APGs, the Department contributes to CAP Goals established by the 
Administration. These CAP Goals are used by leadership to accelerate progress on a limited 
number of Presidential priority areas where implementation requires active collaboration among 
multiple agencies. The Department will contribute to the CAP goals, as appropriate, and will 
include information on its contributions in the FY 2018 Annual Performance Report and FY 2020 
Annual Performance Plan. Additionally, please refer to www.Performance.gov for the agency’s 
contributions to those goals and progress, where applicable. 
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STRATEGIC GOALS AND OBJECTIVES 


Strategic Goal 1 


SIU] 0) oLe) a me-je-lccm- lace Mm (oler-lM-1ice) acm comin) elaen-m(-r-laaliace meleicecelai(-s-m ie) ar-U| 


P-12 students in every community. 


Goal leader: Assistant Secretary for Elementary and Secondary Education 


Strategic Objective 1.1 


Increase high-quality educational options and empower students and 
parents to choose an education that meets their needs. 
Strategic Objective Leader: Assistant Deputy Secretary for Innovation and Improvement 


Implementation Strategies: 


Parents and students need to know about and understand the educational options available to 
them. Further, parents and students should be able to select the educational experience that 
best suits their needs. If every student had the opportunity to pursue personal development in 
the context of an organization aligned with their unique interests, our nation would be stronger 
as a result. To that end, the Department will support greater state and local flexibility in 
elementary and secondary education and encourage SEAs and LEAs to increase the number 
and quality of educational opportunities to provide for meaningful school choice. The two 
primary strategies the Department will use to achieve this strategic objective are the 
administration of federal programs that increase educational options (e.g., the Charter School 
Program (CSP) and the Magnet Schools Assistance Program (MSAP)) and the provision of 
technical assistance. 


Next Steps: 


Several offices across the Department support this Strategic Objective, including the Office of 
Elementary and Secondary Education (OESE), the Office of Innovation and Improvement, the 
Office of Special Education and Rehabilitative Services (OSERS), the Office for Civil Rights 
(OCR), and the Office of English Language Acquisition (OELA). During FY 2018 and FY 2019, 
the Department will: 


e Administer federal programs and use the proposed Secretary’s Supplemental Priority on 
educational choice in discretionary grant programs, when appropriate and consistent 
with a program’s authorizing statute. 

e Develop and disseminate quarterly resources on evidence-based and promising 
practices related to school choice. 

e Deliver relevant technical assistance to states and communities on school choice topics. 

e Apply effective oversight of CSP and MSAP on grant performance. 

e Respond to risks identified by national audits and studies of charter schools and school 
choice. 

e Provide information and technical assistance to support parents, including parents of 
children with disabilities, in choosing a high—quality education option for their child. 

e Provide information to states regarding distributing federal Individuals with Disabilities 
Education Act (IDEA) funds to all LEAs, including charter LEAs. 
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e Provide information and technical assistance to support states in ensuring that the rights 
of children with disabilities are protected when parents exercise school choice. 

e Host a symposium in FY 2018 to explore weighted student funding (WSF) models, 
including lessons learned from early adopters, challenges, and opportunities, to inform 
the Department's work in developing policy guidance and programmatic infrastructure 
that will support efforts by up to 50 districts in the 2018-19 school year to adopt WSF 
systems that improve funding equity and support expanded educational choice. 

e Launch the Flexibility for Equitable Per-Pupil (weighted student funding) demonstration 
program authorized by Title |, Part E of the ESEA. 


Performance Measures: 


1.1.4 | Number of open and operating charter schools supported by CSP. 4B INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
SY 2016-17 SY 2017-18 SY 2018-19 SY 2019-20 
3,549 3,699 Prior Year +150 Prior Year +150 


1.1.A This metric is aligned with an FY 2018-19 APG. The APG focuses on the creation and expansion of new charter schools, which is the 
growth represented in the performance targets for this metric. 


Data Source and Frequency of Collection: National Center for Education Statistics, CCD (used to identify all charter schools 
and enrollees) and grantee annual performance reports (used to identify the subset of charter schools in CCD that receive CSP support); Annually. 


1.1.,B | Number of students enrolled in charter schools supported by CSP. 4B INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
SY 2016-17 SY 2017-18 SY 2018-19 SY 2019-20 
1,539,854 1,564,854 Prior Year +25,000 Prior Year +25,000 


1.1.B This metric is aligned with an FY 2018-19 APG. The APG focuses on the number of students enrolled in new charter schools, which is the 
growth represented in the performance targets for this metric. 


Data Source and Frequency of Collection: National Center for Education Statistics, CCD (used to identify all charter schools 
and enrollees) and grantee annual performance reports (used to identify the subset of charter schools in CCD that receive CSP support); Annually. 


1.1.C Number of new resources on evidence-based and promising practices related to school choice 
disseminated. AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 4 4 4 


1.1.C This metric is aligned with an FY 2018-19 APG. 
Data Source and Frequency of Collection: National Charter School Resource Center and IES sponsored materials; Annually. 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education a 


STRATEGIC GOALS AND OBJECTIVES 


1.1.D Number of students enrolled in magnet schools. AB INCREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
AY 2017-18 Baseline Year AY 2018-19 AY 2019-20 
TBD 75,697 78,824 


1.1.D. Given the changes in the program statute under ESSA, baseline data for this metric begins with the 2017 cohort. The 2017 MSAP cohort 
was awarded funding and began their projects in October of 2017. The fall enrollment for the 2017-2018 school year is considered the 
baseline year for enrollment in the MSAP project schools for the 2017 cohort of grantees. 


Data Source and Frequency of Collection: Grantee annual performance reports; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 1.2 


Provide all P-12 students with equal access to high-quality educational 


opportunities. 


Strategic Objective Leader: Assistant Secretary for Special Education and Rehabilitative 
Services 


Implementation Strategies: 


Every child, regardless of his or her ZIP code or family income, should have access to a high— 
quality education. The implementation of the ESSA, which reauthorized the ESEA, provides a 
framework for states and school districts to provide high—quality educational opportunities that 
meet the needs of their students, based on the following: 


e High academic standards and assessments. 

e A well-rounded education that includes a safe and healthy environment and integrated 
technology. 

e Access to effective teachers and strong systems of collaboration among education 
stakeholders. 

e Proper support and interventions for struggling students and opportunities for enrichment 
experiences before, after, and during the school day. 


This Strategic Objective considers characteristics that are necessary to ensure both access to 
Opportunities and the quality of such opportunities. The Department is committed to improving 
access to high—quality P-12 education opportunities for each student and will support 
educational institutions, parents, and communities in developing such opportunities as well as 
their capacity to improve the outcomes for each student. 
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Next Steps: 


Several offices across the Department support this Strategic Objective, including OESE, 
OSERS, OCR, the Office of Education Technology (OET), the Office of Career, Technical, and 
Adult Education (OCTAE), and the Office of Planning, Evaluation, and Policy Development 
(OPEPD). During FY 2018 and FY 2019, the Department will: 

e Develop accountability system summaries of all approved ESSA state plans for internal 
use to help build staff capacity to support states as they implement and, as necessary, 
refine their plans. The Policy and Program Studies Service (PPSS) within OPEPD will 
develop these summaries based on state plans that the Department has approved. 

e Prepare and disseminate studies on K-12 program or policy issues that could inform or 
support state implementation of ESSA state plans across programs. 

e Provide states, districts, schools, and families: 

o Literacy instructional and assessment strategies and accessible digital 
instructional materials to students who are blind, visually impaired, or who have 
print disabilities. 

o Resources and technical assistance that support preschool children with 
disabilities growth in social-emotional skills, acquisition and use of knowledge 
and skills, and use of appropriate behaviors to meet their needs. 

o Resources and technical assistance that support the participation of children with 
disabilities in early childhood classrooms. 

o Technical assistance on the development, implementation, and sustainability of 
multi-tiered systems of support, and intervention strategies to reduce 
inappropriate school behavior and improve school conditions. 

o Technical assistance to ensure children with disabilities have appropriately 
ambitious individualized education program (IEP) goals and challenging 
objectives. 

o Technical assistance to support improved results for economically disadvantaged 
students, children with disabilities, English learners, migrant, homeless, charter 
schools’ attendees, magnet schools’ attendees, and CTE concentrators. 

e Provide targeted and intensive monitoring and support as well as general technical 
assistance based on risk assessments. 

e Conduct a National Educational Technology Survey to provide national estimates on the 
availability and use of educational technology in elementary and secondary schools. 


Performance Measures: 


1.2.A Percentage of states that show improvement in the percentage of students in grades 3-8 scoring at 
or above proficient on state assessments in reading in all of the following subgroups: economically 
disadvantaged, children with disabilities, English learners, migrant, homeless, and major racial and 

ethnic groups. 


AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
SY 2015-16 SY 2016-17 SY 2017-18 SY 2018-19 
19% 19% 22% 25% 


1.2.A The major racial and ethnic groups are the subgroups defined by states in their state plans under the Elementary and Secondary Education 
Act, as amended. The subgroups may vary by states; states typically report on 5 to 7 racial and ethnic groups. 


Data Source and Frequency of Collection: Assessment Data File that includes state reported data pulled from EDFacts files C175, C178, C185, 
and C188; Annually. 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 1 


STRATEGIC GOALS AND OBJECTIVES 


1.2.B Percentage of states that show improvement in the percentage of students in grades 3-8 scoring at 
or above proficient on state assessments in mathematics in all of the following subgroups: 
economically disadvantaged, children with disabilities, English learners, migrant, homeless, and 
major racial and ethnic groups. 


AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
SY 2015-16 SY 2016-17 SY 2017-18 SY 2018-19 
45% 45% 48% 51% 


1.2.B The major racial and ethnic groups are the subgroups defined by states in their state plans under the Elementary and Secondary Education 
Act, as amended. The subgroups may vary by states; states typically report on 5 to 7 racial and ethnic groups. 


Data Source and Frequency of Collection: Assessment Data File that includes state reported data pulled from EDFacts files C175, C178, C185, 
and C188; Annually. 


1.2.C Percentage of states with 80% or more of preschoolers with disabilities that showed greater than 
expected growth in positive social-emotional skills (including social relationships) by the time they 4B NcREASE 
exit IDEA Part B, section 619 services. 


2017 2018 2019 2020 
FY 2015 FY 2016 FY 2017 FY 2018 
55% 56% 57% 58% 


Data Source and Frequency of Collection: IDEA Part B State Annual Performance Reports; Annually. 


| i) Percentage of schools in the country that have actual Internet bandwidth speeds of at least 


100 Mbps AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
88% 947% 987% 100% 


Data Source and Frequency of Collection: Education Superhighway; Annually. 


1.2.E Percentage of rural schools connected to a broadband infrastructure capable of scaling to 


10 gigabits per second. AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
94% 967% 987% 997% 


Data Source and Frequency of Collection: Education Superhighway; Annually. 
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1.2.F Percentage of states publishing report cards on the preceding school year in a timely manner (i.e., AB INCREASE 
by January 15th of the year following the reporting year). 


Out-Year TARGETS 


2018 2018 2019 2020 
SY 2017-18 Baseline Year SY 2018-19 SY 2019-20 
TBD A greater than or equal A greater than or equal 
percentage of states percentage of states 


publishing timely report publishing timely report 
cards (or maintain 100% cards (or maintain 100% 


if the 2017-18 baseline if the SY 2018-19 
data showed all data showed all 
states publishing) states publishing) 


1.2.F-1 Timeliness is defined as report cards that are published by January 15th of the year following the reporting year (e.g., report cards on the 
2017-18 school year would be timeline if published by January 15, 2019). 


1.2.F-2 Although both of the metrics for timeliness (1.2.F) and completeness (1.2.G) relate to the same school years, there are timing anomalies that 
impact when the data are reported. 


Data Source and Frequency of Collection: Consolidated State Performance Report (CSPR); Annually. 


1.2.G Percentage of monitored states reporting information on each indicator in the state’s accountability 
system and the list of schools identified for comprehensive support and improvement and targeted 4B INCREASE 
support and improvement. 


Out-Year TARGETS 


2019 2018 2019 2020 
SY 2017-18 Baseline Year SY 2018-19 
TBD N/A TBD 


1.2.G Although both of the metrics for timeliness (1.2.F) and completeness (1.2.G) relate to the same school years, there are timing anomalies 
that impact when the data are reported. 


Data Source and Frequency of Collection: OESE Title | monitoring reports of ESSA state report cards; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 1.3 


Prepare all students for successful transition to college and careers by 
supporting access to dual enrollment, job skills development and high- 


quality science, technology, engineering and mathematics (STEM). 


Strategic Objective Leader: Assistant Secretary for Career, Technical and Adult Education 


Implementation Strategies: 


Well-rounded education programs are critical to preparing students for college and careers. 
Access to accelerated coursework to earn postsecondary credit while still in high school, such 
as Advanced Placement and International Baccalaureate courses, dual or concurrent enrollment 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education Es =-— 


STRATEGIC GOALS AND OBJECTIVES 


programs, as well as to STEM opportunities and career and technical education is critical for 
students as they prepare for the transition to college and career and the constantly changing 
career demands of the technology-driven global economy. 


Next Steps: 


Several offices across the Department support this Strategic Objective, including OCTAE, 
OESE, OPEPD, OCR and OSERS. During FY 2018 and FY 2019, the Department will: 

e Provide technical assistance to recipients and subrecipients of funds under: 

o The Adult Education and Family Literacy Act to improve program management 
and participant outcomes. 

o The Carl D. Perkins Career and Technical Education Act to improve program 
management and participant outcomes. 

e Conduct outreach related to the President's Memorandum on STEM education, which 
calls for investing $200 million annually in high-quality STEM education, with an 
emphasis on computer science. 

e Use the proposed Secretary’s Supplemental Priority on STEM in discretionary grant 
programs, when appropriate and consistent with the program’s authorizing statute. 

e Develop guidance and activities regarding transition services and pre-employment 
transition services for students and youth with disabilities. 

e Provide technical assistance to support state special education and Vocational 
Rehabilitative staff to provide transition services and pre-employment transition services 
to youth with disabilities. 

e Utilize the Parent Training and Information Centers to provide parents with technical 
assistance, information, and training about disabilities under IDEA, IDEA rights and 
protections and other relevant laws. 


Performance Measures: 


1.3.4 Number of discretionary grant notices with STEM as a priority. 4B INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
3 8 ») 10 


Data Source and Frequency of Collection: Program offices holding discretionary grant competitions each year (including OESE, Oll, OSERS, 
OPE, OCTAE, IES, and OELA); Annually. 
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1.3.B Number of public high school students by graduating cohort who have taken at least one Advanced 


Placement (AP) Science, Technology, Engineering, Mathematics (STEM) exam while in high school. AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
SY 2015-16 SY 2016-17 SY 2017-18 SY 2018-19 
622,553 653,681 Prior Year + 5% Prior Year + 5% 


1.3.B The following AP exams included in this metric are Biology, Chemistry, Environmental Science, Physics B, Physics C: Electricity and 
Magnetism, Physics C: Mechanics Based, Calculus AB, Calculus BC, Computer Science A, and Statistics. 


The targets indicate a percent increase of 5% over the prior year versus a percentage point increase. 


Data Source and Frequency of Collection: College Board; Annually. 


1.3.C Number of public high school students by graduating cohort who have taken at least one Advanced 
Placement (AP) Science, Technology, Engineering, Mathematics (STEM) exam while in high school and AB INCREASE 
score a 3 or better. 


Out-Year TARGETS 


2017 2018 2019 2020 
SY 2015-16 SY 2016-17 SY 2017-18 SY 2018-19 
339,784 356,773 Prior Year + 5% Prior Year + 5% 


1.3.C The following AP exams in cluded in this metric are Biology, Chemistry, Environmental Science, Physics B, Physics C: Electricity and 
Magnetism, Physics C: Mechanics Based, Calculus AB, Calculus BC, Computer Science A, and Statistics. 


The targets indicate a percent increase of 5% over the prior year versus a percentage point increase. 


Data Source and Frequency of Collection: College Board; Annually. 


1.3.D Percentage of adult education participants in adult basic education programs who achieve a 
measurable skill gain. 


AB INCREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD TBD TBD 


Data Source and Frequency of Collection: National Reporting System (NRS) for Adult Education; Annually. 


1.3.E Percentage of adult education participants who obtain a secondary school diploma or its equivalent and are 


employed or enrolled in an education or training program within one year following exit. AB INCREASE 


Out-Year TARGETS 


2019 2018 2019 2020 
FY 2019 Baseline Year FY 2020 
TBD N/A TBD 


Data Source and Frequency of Collection: NRS for Adult Education; Annually. 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education ayy 


STRATEGIC GOALS AND OBJECTIVES 


1.3.F Percentage of secondary career and technical education (CTE) concentrators who attain a secondary 
school diploma, a General Education Development credential or other state-recognized equivalent 
(including recognized alternative standards for individuals with disabilities), or a proficiency credential, 
certificate, or degree, in conjunction with a secondary school diploma. 


AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2016 FY 2017 FY 2018 FY 2019 
95.81% 96% 96% 96% 


Data Source and Frequency of Collection: State Consolidated Annual Reports (CARs) for the Carl D. Perkins Career and Technical Education 
Act; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 

Strategic Objective 1.4 


Support agencies and institutions in the implementation of evidence- 
based strategies and practices that build the capacity of school staff and 


families to support students’ academic performance. 


Strategic Objective Leader: Director of Policy and Program Studies Service, OPEPD 


Implementation Strategies: 


A greater focus on evidence of what works for educators will help better serve students, 
families, and communities. This Strategic Objective aims to support the development of 
evidence about what works in P-12 education, primarily through expanded support for SEAs 
and LEAs as they implement provisions in the ESSA that require the use of evidence when 
determining what education interventions to implement. 


Next Steps: 


Several offices across the Department support this Strategic Objective, including OPEPD, 
OSERS, OESE and the Institute of Education Sciences (IES). During FY 2018 and FY 2019, the 
Department will: 

e Collaborate with internal and external partners to disseminate resources related to use 
of evidence. 

e Utilize the Parent Training and Information Centers to provide parents with technical 
assistance, information and training about disabilities, IDEA rights and protections, and 
other relevant laws. 

e Support state work related to the State Systematic Improvement Plan (SSIP) to promote 
increased state capacity in evaluating the efficacy, fidelity of implementation and impact 
of evidence-based practices. 

e IES, through its National Center for Education Research and National Center for Special 
Education Research, will: 

o Announce the FY 2018 funding opportunity for Small Business Innovation 
Research, and 
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o Announce FY 2019 funding opportunities for Education Research, Special 
Education Research, and Partnerships and Collaborations Focused on Problems 
of Practice or Policy. 
e Release What Works Clearinghouse products on what is known about the effectiveness 
of various P-12 educational interventions. 
Performance Measure: 


1.4.A Number of technical assistance engagements, events or related activities or products focused on the 


grantees’ use of evidence in kindergarten through grade 12 education. BRINGREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
SY 2017-18 Baseline Year SY 2018-19 SY 2019-20 
TBD Prior Year +15% Prior Year +15% 


1.4.A. The targets indicate a percent increase of 15% over the prior year versus a percentage point increase. 


Data Source and Frequency of Collection: Department of Education offices that deliver technical assistance; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Note on performance metrics and targets: These metrics were established as a part of the 


development of FY 2018-22 Strategic Plan. Metrics may be updated or revised in the future. 
Such updates or revisions will be identified in footnotes. 
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= 2019 President's Budget 


Goal 1 Discretionary Resources 


= 2018 Annualized CR 
= 2017 Appropriation 


Major Discretionary Programs and Activities Supporting Goal 1 


10,000,000 
(Dollars in Thousands) 


20,000,000 


30,000,000 


40,000,000 


: FY 2018 FY 2019 
POC ACCT = on Program A as 20er Annualized President’s 
ppropriation CR Budget 
1.2, | Title | Grants 
OESE ED 1 1.3, | to local 15,386,180 15,428,437 | 15,459,802 
1.4 | educational 
agencies 
oEsE | sip | 1. | 2-2 | State 369,100 366,593 369,100 
1.4 | assessments 
1.1, | Opportunity 
Oll 1&l 1 1.2 | Grants - - 500,000 
(proposed 
legislation) 
Oll iat | 4 | 24, | Charter 342,172 339,848 500,000 
1.2 | schools grants 
1.1, | Special 
OSERS | SE 1 1.2, | education 11,939,805 11,984,380 | 12,002,848 
1.4 | grants to 
States 


Note: On February 9, 2018, the President signed the Bipartisan Budget Act of 2018, which 
includes a two-year cap deal that raises the FY 2019 spending caps significantly. As such, the 
Administration submitted an addendum to its FY 2019 Budget that includes additional funding 
for a limited set of Administration priorities under the new, higher cap levels. Please see 
https:/Awww2.ed.gov/about/overview/budget/budget19/index.html for final levels in the FY 2019 


Budget. 
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Strategic Goal 2 
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Goal leader: Assistant Secretary for Postsecondary Education 
Strategic Objective 2.1 


Support educational institutions, students, parents and communities to 
increase access and completion of college, lifelong learning and career, 


technical and adult education. 


Strategic Objective Leader: Deputy Assistant Secretary for Management and Planning, Office 
of Postsecondary Education (OPE) 


Implementation Strategies: 


The Department recognizes there are multiple, diverse routes to postsecondary education and 
employment and, under this Strategic Objective, seeks to support access to these educational 
Opportunities and their completion for students at all stages of life. To that end, the Department 
will support the educational institutions that advance such opportunities through the 
administration of federal programs and the provision of technical assistance. Additionally, the 
Department will leverage opportunities to enhance the dissemination of information about these 
programs and to work collaboratively with partners to improve institutional oversight. 


The Department will focus on improved utilization of discretionary grant funding so that grantees 
can fully achieve program goals and best serve students. A key indicator of program activity is 
drawdown of funds from grants that have been awarded. The Department will improve utilization 
of federal funding in monitoring the number of grantees with large available balances and 
providing technical assistance to those grantees to remedy such balances. 


A key indicator of successful administration of a grant program is fulfillment of all statutory 
requirements. The Department will monitor compliance with reporting and other requirements 
and assess whether adequate financial and human capital resources are in place to conduct 
mandated activities. 


Next Steps: 


Several offices across the Department support this Strategic Objective, including the OPE, 
OPEPD, FSA, and OCR. During FY 2018 and FY 2019, the Department will: 

e Provide technical assistance to Congress, as requested, to inform the reauthorization of 
the Higher Education Act — such technical assistance includes reviewing draft legislation 
or policies for technical accuracy as well as providing information as to the impact of 
such proposals. 

e Amend Title IX regulations to protect all students from harassment and violence by 
utilizing the federal rulemaking process to seek public comment on proposed 
regulations. 
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e Strengthen grants management practices to maximize the impact of the Department's 
discretionary grants portfolio. 

e Conduct, as appropriate and necessary, negotiated rulemaking to ensure the 
development of regulations that are fair, efficient, and effective for all affected parties. 

e Automate applicable College Scorecard measures within the National Student Loan 
Data System. 

e Release College Scorecard information, including earnings, annually. 

e Promote promising postsecondary enrollment strategies designed to improve 
postsecondary success. 

e Provide resources and tools to parents and students to increase awareness and 
understanding of their rights, and to schools, colleges, and universities to increase 
awareness and understanding of their obligations, under federal civil rights laws by 
increasing the number of OCR technical assistance presentations and releases of public 
information. 

e Promote first-time FAFSA filing among high school seniors, to increase access to 
postsecondary education. 

e Improve the FAFSA-filing experience for first time aid recipients, to improve persistence 
among first-time FAFSA filers. 


Performance Measures: 


2.1.A Percentage of OPE grantees with large available balances. H DECREASE 
| sng Sse ea Out-Year TARGETS 
2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD TBD TBD 


2.1.A. The programs will mostly be the same but in different stages. Year 1 of a program tends to look very different because of startup. The final 
year tends to look very different because a grantee spends down its remaining funds. 


Data Source and Frequency of Collection: The Department's Grants Management System G5; Quarterly. 


2.1.B Percentage of OPE grantees with large available balances that received technical assistance resulting in AB INCREASE 
“Resolved with Good Explanation.” 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD 100% 100% 


Data Source and Frequency of Collection: The Department's Grants Management System G5; Annually. 
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2.1.C Percentage of annual statutory requirements for OPE programs that are fulfilled by OPE. AB INCREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD TBD TBD 


2.1.C Anticipating Congressional reauthorization of the Higher Education Act in FY 2018, the Department cannot anticipate the number or type 
of mandated reporting requirements in the new statute, thereby making the establishment of a baseline not possible until FY 2020 as FY 2019 will 
be a transition year. 


Data Source and Frequency of Collection: The Department's Grants Management System G5; Annually. 


2.1.D Percentage of first-time FAFSA filers among high school seniors AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
60.20% 66.4% TBD TBD 


Data Source and Frequency of Collection: FSA's Central Processing System; Annually. 


2.1.E Persistence among first-time filing aid recipients AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
82.6% 82.6% TBD TBD 


Data Source and Frequency of Collection: FSA’s Common Origination and Distribution system; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 
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Strategic Objective 2.2 


Support agencies and educational institutions in identifying and using 
evidence-based strategies or other promising practices to improve 


educational opportunities and successfully prepare individuals to compete 
in the global economy. 


Strategic Objective Leader: Director of Policy and Program Studies Service, OPEPD 


Implementation Strategies: 


The Department is committed to improving educational opportunities for the existing and future 
workforce. As such, this Strategic Objective focuses on the administration of federal programs 
that provide educational opportunities, training, and support services to the workforce as well as 
the use of evidence. It is also essential that the Department fulfills its commitment to individuals 
with disabilities by working with state and local agencies to provide job-driven training and 
support services, consistent with the Workforce Innovation and Opportunity Act. 


Next Steps: 


Several offices across the Department support this Strategic Objective, including the OPE, 
OPEPD, IES, OSERS, and OCTAE. During FY 2018 and FY 2019, the Department will: 

e Strengthen grants management practices to maximize the impact of the Department’s 
discretionary and formula grants portfolio. 

e Leverage discretionary grant competitions to promote the use of evidence-based and 
promising practices. 

e Launch a CTE research network to increase the number of high-quality, causal studies 
on CTE practices, policies, and programs. 

e Provide technical assistance to states and adult education providers to promote the 
effective implementation of Integrated English Literacy and Civics Education program 
activities that prepare individuals for unsubsidized employment in in-demand industries 
and occupations. 

e Provide assistance to states to increase apprenticeship opportunities for high school 
CTE students. 

e Highlight Career Pathways for Individuals with Disabilities demonstration grant projects 
funded by OSERS. 

e Announce FY 2019 funding opportunities for Education Research and Special Education 
Research and Partnerships and Collaborations Focused on Problems of Practice or 
Policy. 

e Release reports and What Works Clearinghouse products that describe what is known 
about effective programs or practices in postsecondary education. 

e Promote promising postsecondary enrollment strategies designed to improve 
postsecondary success. 
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Performance Measures: 


2.2.A Number of technical assistance events or activities and products focused on the use of evidence in federal 


programs that promote educational opportunities, training, and support services for the workforce. AB INCREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD Prior Year + 10% Prior Year + 10% 


2.2.A The targets indicate a percent increase of 10% over the prior year versus a percentage point increase. 


Data Source and Frequency of Collection: Department of Education offices that deliver technical assistance; Quarterly. 


2.2.B Percentage of adult education program participants who are in unsubsidized employment during the 
second quarter after exit from the program. 


Out-Year TARGETS 


AB INCREASE 


2019 2018 2019 2020 
FY 2019 Baseline Year FY 2020 
TBD N/A TBD 


Data Source and Frequency of Collection: NRS for Adult Education; Annually. 


2.2.C Percentage of VR participants who are in unsubsidized employment during the second quarter after 


exit from the program. 4B INCREASE 


Out-Year TARGETS 


2020 2018 2019 2020 
Program Year (PY) 2020 Baseline Year 
TBD N/A N/A 


Data Source and Frequency of Collection: RSA 911 Vocational Rehabilitation Case Service Report; Quarterly. 


2.2.D Percentage of VR program participants who, during a program year, are in an education or training program 
that leads to a recognized postsecondary credential or employment and who are achieving measurable skill AB INCREASE 
gains, defined as documented academic, technical, occupational, or other forms of progress, towards such 

a credential or employment. 


Out-Year TARGETS 


2020 2018 2019 2020 


PY 2020 Baseline Year 


TBD N/A N/A 


Data Source and Frequency of Collection: RSA 911 Vocational Rehabilitation Case Service Report; Quarterly. 


N/A = Not applicable 
TBD = To be determined 
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Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 

Strategic Objective 2.3 


Support agencies and educational institutions as they create or expand 


innovative and affordable paths to relevant careers by providing 
postsecondary credentials or job-ready skills. 


Strategic Objective Leader: Assistant Secretary for Career, Technical and Adult Education 


Implementation Strategies: 


It is critical that we ensure our nation’s workforce is prepared to meet the challenges of 
tomorrow with the skills and credentials that employers require, which can be achieved by 
creating or expanding innovative paths to recognized postsecondary credentials or obtainment 
of job-ready skills for careers in in-demand industry sectors or occupations. Such credentials 
may be obtained by students at a wide variety of education providers, such as traditional IHEs, 
non-traditional education providers, and other providers of self-guided learning. Through this 
Strategic Objective, the Department will provide grant funding and technical assistance 
resources to develop, evaluate, and replicate practices and programs that expand access to 
viable educational and career pathways that are innovative and affordable and lead to educated 
citizens with quality careers. 


Next Steps: 


Several offices across the Department support this Strategic Objective, including the OPE, 
OCTAE and OSERS. During FY 2018 and FY 2019, the Department will: 

e Provide technical assistance to: 

oO Minority-serving community colleges in forming peer-to-peer communities of 
practice and sharing information. 

o Community colleges in embedding cumulative, industry-recognized credentials 
within technical associate degree programs that enable students to advance 
along a career pathway to different and potentially higher-paying jobs. 

o Recipients and subrecipients of funds under the Adult Education and Family 
Literacy Act to improve program management and participant employment 
outcomes. 

o Improve the CTE and other outcomes of justice-involved young adults (ages 16— 
24) in diversion programs that are alternatives to initial or continued processing in 
the criminal justice system. 

e Assist state vocational rehabilitation agencies to develop the skills and processes 
needed to meet the requirements of the Workforce Innovation and Opportunity Act by 
providing training and technical assistance through various grants: (1) Workforce 
Innovation Technical Assistance Center; (2) Vocational Rehabilitation Technical 
Assistance Center — Youth with Disabilities; (3) Vocational Rehabilitation Technical 
Assistance Center — Targeted Communities; and (4) Vocational Rehabilitation Agency 
Program Evaluation and Quality Assurance. 

e Support IHEs to provide support to individuals to obtain a degree in rehabilitation 
counseling. 
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Performance Measures: 


2.3.A Number of technical assistance activities, sponsored by the Department, intended to expand or enhance 
the integration of workforce preparation activities within academic instruction in adult education classrooms. 


AB INCREASE 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
3 6 12 12 


Data Source and Frequency of Collection: Quarterly Progress Reports from Contractor; Quarterly 


2.3.B Percentage of adult education participants enrolled in an integrated education and training (IET) program. AB INCREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD TBD TBD 


Data Source and Frequency of Collection: NRS for Adult Education; Annually. 


2.3.C Percentage of postsecondary CTE concentrators who received an industry-recognized credential, a 


; INCREASE 
certificate, or a degree. t 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2016 FY 2017 FY 2018 FY 2019 
54.9% 55% 56% 56% 


Data Source and Frequency of Collection: State Consolidated Annual Reports (CARs) for the Car! D. Perkins Career and Technical Education 
Act; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 2.4 


Improve quality of service for customers across the entire student aid life 


cycle. 


Strategic Objective Leader: Chief Operating Officer, Federal Student Aid (FSA) 


Implementation Strategies: 


To improve the way the Department serves its customers, the Department will modernize the 
loan-servicing environment by taking the best ideas and capabilities available and putting them 
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to work for Americans with federal student loans. This Strategic Objective focuses on the 
customer service provided by the Department to students, parents, borrowers, and institutions. 
The Department will deliver a best-in-class processing and servicing environment capable of 
sustaining the management of the growing federal student loan portfolio. 


Next Steps: 


During FY 2018 and FY 2019, the FSA will: 

e Ensure accurate and timely information is available and promoted through the 
Department's multiple engagement and information dissemination channels. 

e Provide information geared toward the needs of students at each stage of the student 
aid life cycle. 

e Oversee and monitor non-default federal student loan servicers to ensure they meet the 
requirements in their contracts. 

e Ensure the Department’s non-default federal student loan servicers have websites that 
borrowers can use to log on, view their loan information, make a payment, and 
complete other transactions. 

e Ensure the Department's non-default federal student loan servicers communicate with 
borrowers who are in repayment and are past due. 

e Conduct outreach efforts to student loan borrowers and perform listening sessions of 
customer-call center interactions to ascertain areas for continual improvement in service 
and the customer experience. 


Performance Measures: 


2.4.A Number of non-default federal loan servicers’ call centers with expanded hours. AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 
0 All nine servicers will All servicers will N/A 
offer expanded hours offer expanded hours 
(7 am to 11 pm ET). (7am to 2 am ET). 


2.4.A This metric is aligned with an FY 2018-19 APG. 
Data Source and Frequency of Collection: Federal servicers quarterly reports; Quarterly. 


2.4.B Number of call centers that meet or exceed the quality standard for average speed to answer (ASA). AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 
Average monthly All nine servicers All servicers N/A 
ASA was 76.9 seconds. will meet or exceed will meet or exceed 
ASA less than or ASA less than or 
equal to 70 seconds. equal to 60 seconds. 


2.4.B This metric is aligned with an FY 2018-19 APG. The ASA quality standard improves (i.e. decreases) every year. However, overall the metric 
is to increase the number of centers that meet or exceed the quality standard. 


Data Source and Frequency of Collection: Federal servicers quarterly reports; Quarterly. 
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2.4.C Number of call centers that meet or exceed the quality standard for average abandon rate (AR) for 


: : 4B INCREASE 
incoming calls. 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 
Average monthly All nine servicers All servicers N/A 
AR was 3.8%. will meet or will meet or 
exceed AR less than exceed AR less than 
or equal to 3.50%. or equal to 3.00%. 


2.4.C This metric is aligned with an FY 2018-19 APG. The AR quality standard improves (i.e. decreases) every year. However, overall the metric is to 
increase the number of centers that meet or exceed the quality standard. 


Data Source and Frequency of Collection: Federal servicers quarterly reports; Quarterly. 


2.4.D ACSI Aid Life Cycle Surveys AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
69.9% 69.9% TBD TBD 


Data Source and Frequency of Collection: FSA's Customer Satisfaction Surveys; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 2.5 


Enhance students’ and parents’ ability to repay their federal student 


loans by providing accurate and timely information, relevant tools and 
manageable repayment options. 


Strategic Objective Leader: Chief Operating Officer, FSA 


Implementation Strategies: 


A comprehensive framework for default prevention that allows students to understand and 
access information about college options and associated costs, loan counseling and guidance, 
support for retention, loan repayment options and borrower benefits is critical. The Department 
will develop materials that clearly communicate students’ and parents’ options to repay federal 
student loans before and during the course of pursuing a postsecondary credential. 
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Next Steps: 


Several offices across the Department support this Strategic Objective, including FSA and OPE. 
During FY 2018 and FY 2019, the Department will: 
e Monitor and report on non-default federal student loan servicers’ performance. 

Focus on reducing the number of delinquent borrowers. 

Address Borrower’s Defense regulations. 

Address Gainful Employment regulations. 

Maintain and improve the Studentaid.gov, FinancialAidToolkit.ED.gov and 

CollegeCost.ED.gov websites. 

e Leverage our digital engagement assets to inform, motivate and respond to students, 
parents, and financial aid professionals on the topic of student loans. 

e Engage with financial aid professionals to provide training and information on the tools, 
resources, and information they need to be successful in counseling current and 
prospective student loan borrowers. 

e Partner with minority serving institutions to help increase the financial aid acumen of 
students and parents. 

e Ensure student borrowers receive exit counseling when they complete their degree 
program, withdraw, or enroll at less than half time. 


Performance Measures: 


2.5.A Percentage of borrowers that are more than 30 days delinquent. @& DECREASE 
banner oS ee 
2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
18.4% 18.05% Achieve a 0.5% Achieve a 0.5% 
reduction in borrowers reduction in borrowers 
that are more than 30 that are more than 30 
days delinquent. days delinquent. 


Data Source and Frequency of Collection: FSA’s Data Warehouse, Annually. 


2.5.B Percentage of borrowers that are more than 90 days delinquent. DECREASE 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
10.9% 10.7% Achieve a 0.2% Achieve a 0.2% 
reduction in borrowers reduction in borrowers 
that are more than 90 that are more than 90 
days delinquent. days delinquent. 


Data Source and Frequency of Collection: FSA’s Data Warehouse; Annually. 
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N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Note on performance metrics and targets: These metrics were established as a part of the 
development of FY 2018-22 Strategic Plan. Metrics may be updated or revised in the future. 
Such updates or revisions will be identified in footnotes. 

Goal 2 Discretionary Resources 


Goal 2 Discretionary Resources 


= 2019 President's Budget 
m™2018 Annualized CR 


= 2017 Appropriation 


10,000,000 20,000,000 30,000,000 40,000,000 
(Dollars in Thousands) 


Major Discretionary Programs and Activities Supporting Goal 2 


: FY 2018 FY 2019 
POC ACCT — Program A ecatlew Annualized President’s 
Pprop CR Budget 


2.1, | Federal Pell 
FSA SFA 2 2.3 | grants: 22,475,352 22,322,722 | 22,475,352 
Discretionary 


2.1, | Career and 
OCTAE | CTAE 2 2.3 | technical 1,112,226 1,115,380 1,117,598 
education 
State grants 
2.1 | Strengthening 


OPE HE 2 HBCUs 244,694 243,032 244,694 
2.1, | Federal TRIO 
OPE HE 2 2.2, | programs 950,000 943,549 550,000 


2.3 


Note: On February 9, 2018, the President signed the Bipartisan Budget Act of 2018, which 
includes a two-year cap deal that raises the FY 2019 spending caps significantly. As such, the 
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Administration submitted an addendum to its FY 2019 Budget that includes additional funding 
for a limited set of Administration priorities under the new, higher cap levels. Please see 


https:/Awww2.ed.gov/about/overview/budget/budget19/index.html for final levels in the FY 2019 
Budget. 
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Strategic Goal 3 


Sl ig claleital-lamaatome ler Uiiavam-lerer-x-t-J 10] [1avar-laleMUl-y-me) mre lUcer-lalelamer-ic-} 


alcelelelamol-anc-lamsar-Uar-\el-1iil-al em laced e-y-toy-re mm el antc-CeaVam el ge) ected (elalcw-lale, 
transparency. 


Goal leader: Assistant Secretary for Planning, Evaluation, and Policy Development 
Strategic Objective 3.1 


Improve the Department’s data governance, data life cycle management 


and the capacity to support education data. 


Strategic Objective Leader: Associate Commissioner, Administrative Data Division, National 
Center for Education Statistics, IES 


Implementation Strategies: 


This Strategic Objective focuses on the data that the Department collects and opportunities to 
enhance the Department's data management framework and internal capacity. The main goal of 
improving data management is to clarify the roles and procedures used to optimize data 
integrity and quality. The strategies behind this objective guide the Department’s activities to 
develop and support the highly skilled workforce capacity necessary to execute these 
improvements. As a result, parents, communities, and the Department’s state and local partners 
will be more inclined to make more informed decisions on behalf of their children and 
communities. 


Next Steps: 


Several offices across the Department support this Strategic Objective, including IES, OPEPD, 
and the Office of the Chief Information Officer. During FY 2018 and FY 2019, the Department 
will: 

e Assess the current maturity of data management practices within the Department’s 
program offices and work with program offices to identify and prioritize areas of 
improvement. 

e Deliver training curricula for data professionals within the Department to continue to 
develop a highly skilled workforce throughout the Department to facilitate the accurate 
and appropriate use of data. 

e Explore improvements in the Department’s operating structure that would facilitate better 
data practice. 

e Improve the Department's data governance, data life management, and the capacity to 
support education data through varied efforts that include developing an open source 
data analysis software package called EdSurvey (currently available for NAEP data) for 
international assessment data. 
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Performance Measures: 


3.1.A Number of data management activities for which Department-wide procedures or templates have been | AB INCREASE 
created and reviewed through the Data Strategy Team (DST). | 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 3 6 8 


Data Source and Frequency of Collection: Minutes from Monthly Data Strategy Team (DST) meetings; Quarterly. 


3.1.8 Number of Department program offices participating in DST offered data management training. | 4B INCREASE 
Out-Year TARGETS 
2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
11 18 20 22 


Data Source and Frequency of Collection: Minutes from Monthly DST meetings; Quarterly. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 3.2 


Strategic Objective Leader: Chief Privacy Officer, Office of Management (OM) 


Implementation Strategies: 


The Department is committed to student privacy. While education data have transformative 
potential, the vast amount and sensitivity of these data make it imperative that the Department 
and educational institutions that maintain student data take steps to protect the data adequately. 
This Strategic Objective focuses on improving privacy protections through the administration of 
the Family Educational Rights and Privacy Act (FERPA), the development and dissemination of 
privacy and security training, and conducting technical assistance to SEAs, LEAs, and IHEs. 
The APG for this goal focuses on data security at institutions of higher education, using auditing, 
training, and reporting as tools. 
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Next Steps: 


Several offices across the Department support this Strategic Objective, including the Office of 
the Chief Privacy Officer (OCPO), FSA, IES, Office of the General Counsel (OGC) and all of the 
Department’s program offices. During FY 2018 and FY 2019, the Department will: 
e Administer FERPA, with a focus on improving the timeliness of both enforcement actions 
and policy determinations. 


e Provide needed assistance to educational institutions via the Privacy Technical 
Assistance Center. 


e Provide support, including outreach and compliance efforts, to external partners. 


Performance Measures: 


3.2.A Number of Institutions of Higher Education (IHE) that have an audit of GLBA-related information security 


S| 
safeguards which result in no significant findings Ap incense 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 18 36 77 


3.2.A This metric is aligned with an FY 2018-19 APG. 


Data Source and Frequency of Collection: |HE-provided auditor reports; Quarterly. 


3.2.8 Number of outreach activities targeting data privacy and IT security requirements of institutions of 
; ; AB INCREASE 
higher education. 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
12 14 17 20 


3.2.B This metric is aligned with an FY 2018-19 APG. 


Data Source and Frequency of Collection: Outreach activity records maintained by the Privacy Technical Assistance Center; Quarterly. 


3.2.C Percentage of LEA websites from statistically representative sample reviewed for inclusion of transparency 


INCREA: 
best practices and compliance with legal requirements relating to third party contracting. g Basaiare 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD 35% 60% 


Data Source and Frequency of Collection: Selected LEA public websites; Quarterly (except for FY 2018 while the baseline is being established). 
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N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 3.3 


Increase access to, and use of, education data to make informed 


decisions both at the Department and in the education community. 


Strategic Objective Leader: Director of Policy and Program Studies Service, OPEPD 


Implementation Strategies: 

For education data to be used to support informed decisions, the Department will need to 
develop and implement methods to analyze, visualize, interpret, and represent education data 
and support education stakeholders in doing the same. This Strategic Objective focuses on 
increasing access to education data at all levels, and improving the tools necessary to support 
the appropriate use of education data for decision-making by the Department and education 
stakeholders. 


Next Steps: 


Several offices across the Department support this Strategic Objective, including IES, OPEPD, 
OET, OCR, OCPO, and OSERS. During FY 2018 and FY 2019, the Department will: 

e Improve the usefulness of the NAEP data for informed decision-making in the education 
community through analysis and reporting of a new type of data produced by the new 
NAEP digitally based assessments. The new digital format produces data on actions 
students take in responding to assessment questions. 

e Conduct outreach and provide support to LEAs to prepare for the 2017-18 CRDC data 
submission period to increase data access and use. 

e Review data provided by grant recipients and provide technical assistance on data 
quality and use. 

e Expand the InformED initiative to further develop the Department’s open data 
infrastructure. 

e Ensure Accountability and Audit Resolution Tracking System (AARTS) system data are 
secure, accurate and reliable. 

e Improve access to NCES’ extensive sample-survey data by integrating nine additional 
data sets on topics ranging from early childhood to postsecondary education into its 
public facing Datalab tool. 

e Provide training, coaching and applied research assistance to SEA and LEA staff 
through the Regional Educational Laboratory program to assist with the analysis of 
education data. 

e Increase the Department’s awareness and appropriate use of: 

o The Open Licensing Requirement for Competitive Grant Programs and 
associated exemptions. 

o Data dissemination requirements in grants competitions. 

o Sessions dedicated to improved data use for external audiences. 

e Complete a feasibility study on improvements to the open data IT infrastructure at the 
Department. 
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e Ensure current links to machine readable data sources in the Public Data Listing are 
accurate and complete. 

e Encourage offices to include releasing data in a machine-readable format into their data 
dissemination plans. 


Performance Measures: 


3.3.A Number of sessions dedicated to improved data use provided to external grantees and stakeholders 


| 
presented by Department employees or their contractors, or occurring at Department hosted events. BRINGS: 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD 4 6 


Data Source and Frequency of Collection: PPSS records; Quarterly. 


3.3.8 Number of newly added publicly available datasets in machine readable formats. | AB INCREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD 8 12 


Data Source and Frequency of Collection: The Department's Public Data Listing; Quarterly. 


3.3.C Percentage of discretionary grant competitions that include the requirement to openly license to the public 
copyrightable grant deliverables created with Department grant funds. 


Out-Year TARGETS 


AP INCREASE | 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD 80% 90% 


Data Source and Frequency of Collection: Information Collection Packages; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Note on performance metrics and targets: These metrics were established as a part of the 


development of FY 2018-22 Strategic Plan. Metrics may be updated or revised in the future. 
Such updates or revisions will be identified in footnotes. 
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Goal 3 Discretionary Resources 


Goal 3 Discretionary Resources 


m= 2019 President's Budget 
= 2018 Annualized CR 
= 2017 Appropriation 


- 100,000 200,000 300,000 400,000 500,000 600,000 
(Dollars in thousands) 


Major Discretionary Programs and Activities Supporting Goal 3 


. FY 2018 FY 2019 
POC ACCT eae a Program A a ae Annualized President’s 
ppropriation CR Budget 
3.1, | Research, 
IES IES 3 3.2, | development, 187,500 186,227 187,500 
3.3 and 
dissemination 
3.1, 
IES IES 3 3.2, | Statistics 109,500 108,756 112,500 
3.3 
3.1, National 
IES IES 3 3.2, | assessment 149,000 147,988 149,000 
3.3 


Note: On February 9, 2018, the President signed the Bipartisan Budget Act of 2018, which 
includes a two-year cap deal that raises the FY 2019 spending caps significantly. As such, the 
Administration submitted an addendum to its FY 2019 Budget that includes additional funding 
for a limited set of Administration priorities under the new, higher cap levels. Please see 


https://www2.ed.gov/about/overview/budget/budget19/index.html for final levels in the FY 2019 
Budget. 
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Strategic Goal 4 


Reform the effectiveness, efficiency and accountability of the 


Department. 


Goal Leader: Assistant Secretary for Management 
Strategic Objective 4.1 


Provide regulatory relief to educational institutions and reduce burden by 
identifying time-consuming regulations, processes and policies and 


working to improve or eliminate them, while continuing to protect 
taxpayers from waste and abuse. 


Strategic Objective Leader: Senior Counselor to the Secretary 


Implementation Strategies: 


Reducing regulatory burden on external stakeholders and improving internal decision-making 
processes will help to ensure greater efficiencies in Department operations and more effective 
and efficient service to the public. 


Next Steps: 


Given the importance of protecting taxpayers from waste and abuse, all Department offices 
support this Strategic Objective. During FY 2018 and FY 2019, the Department will continue to 
identify overly burdensome, outdated, unnecessary, or ineffective regulations and guidance that 
could be improved or eliminated. 


Performance Measures: 


4.1.A Number of evaluations to identify potential EO 13771 deregulatory actions that included opportunity for 


public input and/or peer review. AB INCREASE 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 6 74 TBD 


Data Source and Frequency of Collection: Report to OMB; Biennially. 
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4.1.B | Number of EO 13771 deregulatory actions recommended by the Regulatory Reform Task Force to the 
agency head, consistent with applicable law. 


AB INCREASE 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 13 8 TBD 


Data Source and Frequency of Collection: Department Records; Biennially. 


4.1.C Number of EO 13771 deregulatory actions issued that address recommendations by the Regulatory 


Reform Task Force. AB INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 13 8 TBD 


Data Source and Frequency of Collection: Federal Register and ed.gov; Biennially. 


4.1.D Number of EO 13771 regulatory actions and, separately, EO 13771 deregulatory actions issued. BP INCREASE 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 TBD TBD TBD 


Data Source and Frequency of Collection: Federal Register and ed.gov; Biennially. 


4.1.E Total incremental cost of all EO 13771 regulatory actions and EO 13771 deregulatory actions (including 


; ; INCREASE 
costs or cost savings carried over from previous fiscal years). t 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 Annual Cost -$3 million TBD TBD 
Present Value Cost 
-$42.9 million 


Data Source and Frequency of Collection: Report to OMB; Annually. 
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4.1.- | Number of significant deregulatory actions submitted to OMB, the number of actions reflected 


AB INCREASE 
on reginfo.gov. 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 2 2 N/A 


4.1.F This metric is aligned with an FY 2018-19 APG. 


Data Source and Frequency of Collection: Reginfo.gov; Quarterly. 


4.1.G Number of requests for significance determination of deregulatory actions submitted to OMB, the number 


Ni 
of such requests submitted to OMB by e-mail. Sie 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
0 13 8 N/A 


4.1.G This metric is aligned with an FY 2018-19 APG. 


Data Source and Frequency of Collection: E-mails between the Department and OMB; Quarterly. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 4.2 


Identify, assess, monitor and manage enterprise risks. 


Strategic Objective Leader: Chief Financial Officer 


Implementation Strategies: 


Enterprise Risk Management (ERM) includes integrating existing risk management processes in 
day-to-day and strategic decision-making activities and creating a risk-aware culture. The 
Department will strengthen its ERM practices and foster open discussions about how those 
risks might impact the accomplishment of the Department’s mission and whether the 
Department’s allocation of resources is aligned to best mitigate risks. 


Next Steps: 


Given its focus on enterprise risk, all Department offices support this Strategic Objective. During 
FY 2018 and FY 2019, the Department will: 


e Review and update the enterprise risk portfolio and develop an ERM framework aligned 
to the new Strategic Plan that incorporates other strategic risks as applicable. 
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e Assign accountability for managing enterprise risks to appropriate Department offices 
and senior officials. 

e Integrate actions taken to address management challenges into the enterprise risk 
portfolio and in discussions about how they impact strategic objectives and performance. 

e Develop and maintain an enterprise risk dashboard with leading indicators to inform risk 
management discussions. 

e Hold routine Senior Management Council meetings to discuss key ERM documents, 
including the framework, profile, and dashboards, and how enterprise risks relate to 
management priorities, resource allocation decisions, and other critical issues such as 
cost estimates for student loans. 

e Take actions to further strengthen the integration of the Department and FSA ERM 
efforts, leveraging consistent approaches to identify, assess, and determine mitigation 
strategies for enterprise risks. 

e Improve the maturity level score of the Department’s Information Technology (IT) 
Security Program and Practices as they relate to the Federal Information Security 
Modernization Act of 2014. 

e Take actions to integrate the Department’s ERM framework with the Internal Control 
Program. 

e Implement a “Risk Management Tool-Kit” website on ed.gov to assist external grant 
recipients and staff in identifying, mitigating, and managing risks. 


Performance Measure: 


4.2.A Percentage of key indicators that show improvement in performance and risk management. AB INCREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
TBD TBD TBD 


Data Source and Frequency of Collection: Office of the Chief Financial Officer will calculate using data from various sources; Quarterly. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 4.3 


Strengthen the Department’s cybersecurity by enhancing protections for 


its information technology infrastructure, systems and data. 


Strategic Objective Leader: Chief Information Officer 
Implementation Strategies: 


Improved cybersecurity will be a key contributor in ensuring the Department's systems and data 
are protected, which will help build a strong foundation for the Department’s information 
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technology (IT) infrastructure. As such, the Department will provide proactive cybersecurity 
services, monitor and enhance threat intelligence capabilities, explore shared services and 
cloud capabilities, and improve its cybersecurity workforce. 


Next Steps: 


Given its focus on cybersecurity, all Department offices support this Strategic Objective. During 
FY 2018 and FY 2019, the Department will: 

e Improve the maturity level score of the Department’s IT Security Program and Practices 
as they relate to the Federal Information Security Modernization Act of 2014. 

e Assess the Department's IT inventory. 

e Lead the Education Department Acquisition Regulations Program, focusing on IT to 
strengthen and enhance cybersecurity and support Federal Information Technology 
Acquisition Reform Act (FITARA) compliance. 

e Ensure all public internet connections are compliant with the Department of Homeland 
Security Trusted Internet Connections Reference Architecture, as amended. 


Performance Measure: 


4.3.A Maturity level score of the Department's Information Technology (IT) Security Program and Practices as 


INCREASE 
they relate to the Federal Information Security Modernization Act of 2014 (FISMA). t 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
Identify (Level 3) 60% of security functions 100% of security functions 20% of security functions 
Protect (Level 2) scored at or above scored at or above scored at Level 5 - 
Detect (Level 2) Level 4 - Managed and Level4-—Managed and Optimize and 80% remain 
Respond (Level 2) Measurable Measurable at or above Level 4 - 
Recover (Level 2) Managed and Measurable 


Data Source and Frequency of Collection: Annual (FY) Inspector General Federal Information Security Modernization Act of 2014 (FISMA) 
Reporting Metrics Audit Report; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Strategic Objective 4.4 


Improve the engagement and preparation of the Department's workforce 


using professional development and accountability measures. 


Strategic Objective Leader: Chief Human Capital Officer, Office of Management 
Implementation Strategies: 


In order to best serve the public, the Department must maximize employee effectiveness 
through robust employee engagement and performance management. The Department will 
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continue to build the skills and knowledge of its workforce and will focus on all employees and 
offices. Furthermore, the Department must focus on ways to thrive in an operational climate that 
is significantly resource constrained. This will spur opportunities to involve employees in thinking 
differently about strategies to reengineer, streamline, or even eliminate work that does not serve 
the efficiency of the Department in achieving its strategic goals and desired outcomes. 


Next Steps: 


Given its focus on the Department's workforce, all Department offices support this Strategic 
Objective. During FY 2018 and FY 2019, the Department will: 

e Implement a comprehensive internal communications strategy focused on employee 
engagement. 

e Expand professional development opportunities to further develop workforce skills by 
finalizing competency assessments for mission critical occupations and prioritizing 
training needs, developing a pilot performance management training course to address 
common performance challenges and launching an Administrative Professionals 
Development program. 

e Ensure that all supervisors, managers, and executives receive continual training and 
development to be successful in their roles as federal supervisors and to enhance their 
leadership skills. 

e Restructure the Department’s workforce through enhanced succession planning. 

e Develop strategies to leverage and retain critical employee skills through reassignments, 
on-the-job training, needs assessment and skills gap closure, and enhanced 
professional development opportunities. 


Performance Measures: 


44.A _FEVS employee engagement index score. AB INCREASE 
2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
67% 68% 69% 70% 


Data Source and Frequency of Collection: Office of Personnel Management (OPM) Federal Employee Viewpoint Survey (FEVS); Annually. 


44.B Percentage of positions with competencies identified. AB INCREASE 


Out-Year TARGETS 


2018 2018 2019 2020 
FY 2018 Baseline Year FY 2019 FY 2020 
19% 25% 32% 


Data Source and Frequency of Collection: Talent Management System (TMS); Annually. 
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44.C Percentage of supervisors and managers with a performance plan critical element related to employee 


INCREASE 
engagement. t 


Out-Year TARGETS 


2017 2018 2019 2020 
FY 2017 FY 2018 FY 2019 FY 2020 
5% 35% 50% 80% 


Data Source and Frequency of Collection: TMS and USA Performance; Annually. 


N/A = Not applicable 

TBD = To be determined 

Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Note on performance metrics and targets: These metrics were established as a part of the 
development of FY 2018-22 Strategic Plan. Metrics may be updated or revised in the future. 
Such updates or revisions will be identified in footnotes. 


Goal 4 Discretionary Resources 


Goal 4 Discretionary Resources 


= 2019 President's Budget 
= 2018 Annualized CR 
= 2017 Appropriation 


- 100,000 200,000 300,000 400,000 500,000 600,000 700,000 
(Dollars in thousands) 


Major Discretionary Programs and Activities Supporting Goal 4 


: FY 2018 FY 2019 
POC ACCT a ee J Program A wa sae Annualized President’s 
ppropriation CR Budget 
Program 
ALL DM/PA 4 Administration 432,000 429,066 432,506 
Salaries and 
Expenses 
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FY 2018 FY 2019 
POC ACCT fan sh Program A Bieri Annualized President’s 

PProp CR Budget 
OcR | OcR | 4 a ci. 108,500 107,763 107,438 


Note: On February 9, 2018, the President signed the Bipartisan Budget Act of 2018, which 
includes a two-year cap deal that raises the FY 2019 spending caps significantly. As such, the 
Administration submitted an addendum to its FY 2019 Budget that includes additional funding 
for a limited set of Administration priorities under the new, higher cap levels. Please see 


https:/Awww2.ed.gov/about/overview/budget/budget19/index.html for final levels in the FY 2019 
Budget. 
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Management 


Challenge 


Improper 
Payments 


Major Management Priorities and Challenges 


As summarized in the FY 2017 Agency Financial Report, the Office of Inspector General has 
identified four management challenges for the Department in FY 2018. These challenges are 
detailed in the FY 2018 Management Challenges Report. The Government Performance and 
Results Act Modernization Act of 2010 requires the Department to include in its APP the 
following information on those planned actions, including performance goals, indicators, and 
milestones, to address these challenges: 


PNXoxerolU lal ez l0) (=) 
Official 


Chief Financial 
Officer (FSA) 


Accomplishments 


Obtained approval from OMB for 
alternative improper payment 
estimation methodology for 2017, 
which included refinements over 
prior years 

-Updated the grant and contract risk 
assessment processes to include 
consideration of all required risk 
factors 

-Implemented security control 
improvements and reactivated the 
Data Retrieval Tool (DRT) to enable 
applicants to securely obtain 
necessary financial information to 
complete the FAFSA or apply for an 
income-driven repayment (IDR) 
plan 

-Documented and tested thousands 
of entity-level, process and 
information technology system 
controls, including more than 500 
controls designed to prevent or 
detect improper payments; 
determined that 96% of controls are 
designed and operating effectively, 
with 4% that have immaterial 
deficiencies that are being 
corrected 

-FSA Program Compliance 
performed ongoing program 
reviews of more than 150 schools 
to ensure compliance and identify 
root cause issues to inform 
guidance for schools 


md relay atsxe my Qeat (0) ats) 


Continue to coordinate 
with OMB on 


refinements to 
estimation methodology 
for student aid programs 
-Develop a statistically- 
valid estimation 
methodology 
-Continuously monitor 
processes and systems 
to strengthen controls, 
including work with IRS 
to implement system 
upgrades 

-Maintain improper 
payment compliance 
requirements as a 
significant enterprise risk 
in FSA’s risk profile 


Performance Goals, 
Tate tCers\ ce) comers ale 
Milestones 


Goal: Minimize the risk 
of improper payments 
without unduly 
burdening students 
-Indicator: % improper 
payments 

Key milestones: 
Integrate improper 
payment risk 
management 
strategies with 
updated enterprise risk 
portfolio aligned with 
Strategic Plan 


MAJOR MANAGEMENT PRIORITIES AND CHALLENGES 


Management 


Challenge 


Information 
Technology 
Security 


PNXexerolU lal ez l0) (=) 
fe) ier 


Chief 
Information 
Security 
Officer (OCIO) 


Accomplishments 


Completed the Department’s 
cybersecurity workforce capability 
assessment to identify current gaps 
in Department cybersecurity 
workforce skills and certifications 
-Completed the Department’s 
Cybersecurity Workforce 
Development Plan to address gaps 
identified in the Department's 
cybersecurity workforce capability 
assessment and completed 
implementation plan for coding of 
all cyber positions 

-Established Department/FSA 
Cybersecurity Committee 
Developed a Departmental 
Cybersecurity Strategy and 
Implementation Plan (ED-CSIP) 
-Submitted Cybersecurity 
Framework Implementation Plan to 
OMB, as required by President's 
Executive Order 

-Completed major updates to the 
Department's primary cybersecurity 
policy guidance 

-Re-validated the list of the 
Department's High Value Assets 
(HVAs), to include plans of actions 
to strengthen security controls for 
all HVAs 


ed relay arsxe ws \eat (0) ats) 


Improve the maturity 
level score of the 
Department's 
Information Technology 
(IT) Security Program 
and Practices as they 
relate to the Federal 
Information Security 
Modernization Act of 
2014 (FISMA) 

-Maintain information 
technology security as a 
significant enterprise risk 
in FSA’s risk profile 


Performance Goals, 
Tate tCors\ te) comms tale 
Milestones 


Goal: Mature the 
Department’s 
cybersecurity 
operations to achieve 
the next level of 
effectiveness as 
defined by the FISMA 
maturity model 
-Indicator: 60% of 
security functions will 
be scored at or above 
Maturity Level 4 — 
Managed and 
Measurable 

-Key milestones: 
Calibrate IT security 
risks with other 
enterprise risks in ED’s 
updated risk profile 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 


54 


MAJOR MANAGEMENT PRIORITIES AND CHALLENGES 


PNXexerolU lal ele) (=) 
(ei etl 


Management 


Challenge 


Oversight and | Chief 

Monitoring Enterprise 
Risk Officer 
(FSA) 


Accomplishments 


ED announced a stronger approach 
to FSA compliance enforcement, 
creating stronger consumer 
protections for students, parents 
and borrowers against "bad actors" 
-FSA’s Annual Risk Assessment 
was the first version of the system 
to be built on the EDWA platform, 
increasing efficiency and accuracy, 
and allowing for greater flexibility in 
scoring in future years 

-Initiated study to determine if 
manipulation of financial statement 
composite scores is a widespread 
issue impacting FSA risk 
assessments 

-FSA revamped REACH 
agreements for monitoring to align 
with organization performance and 
improvement efforts including timely 
determinations and improved 
customer service 

-Grant-making offices developed a 
standard discretionary grant site 
visit monitoring tool and related 
training 

-Implemented new cross-program 
risk tools including TAGR and 
Grantworks 

-OPE included periodic random 
checks of grant files in managers’ 
REACH agreements 

-OESE expanded fiscal monitoring 
pilot across programs 


ed Felayarsxe Xen (0) ats) 


Increase expertise 
among Department staff 
to effectively monitor 
student aid program 
participants and grant 
recipients 

-Improve monitoring and 
support processes, 
including increased 
information-sharing 
across offices and more 
integrated and targeted 
monitoring approaches 
-Expand data analysis 
capabilities and continue 
risk-based programmatic 
and fiscal monitoring of 
grant recipients, 
providing assistance in 
meeting requirements 
-Highlight areas of 
ambiguity or common 
misunderstanding for 
program participants and 
provide additional 
technical assistance in 
those areas 

-Maintain monitoring and 
oversight as a significant 
enterprise risk in FSA’s 
risk profile 


Performance Goals, 
Tate tCors\ ce) comms tale 
Milestones 


Goal: Support 
agencies and 
educational institutions 
to improve outcomes 
for students 

-Indicator: % grants 
not making substantial 
progress 

-Key milestones: 
update risk 
assessments for 
student aid 
participants; gather 
feedback from annual 
student aid conference 
attendees; calibrate 
oversight and 
monitoring risks with 
other enterprise risks 
in ED’s updated risk 
profile 
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Management 


Challenge 


Data Quality 
and Reporting 


PNXexexol Ul ales lo) (=) 
(@) iter 


Director, Risk 
Management 
Service (ODS) 


Accomplishments 


OESE and the Office of Special 
Education Program applied the new 
SharePoint tracking tool to 
collaboratively document and 
follow-up on data quality concerns 
identified in the initial submissions 
of the 2015-16 Consolidated State 
Performance Reports (CSPR), with 
a focus on academic assessments 
and graduation rates, which are 
shared areas of stewardship and 
programmatic use 

-EDFacts state Submission plans 
were modified to allow states to 
proactively identify issues affecting 
data submissions 

-EDFacts Data Quality Summary 
Reports can be generated on every 
file collected in EDFacts; Summary 
Reports present high level 
information about timeliness, 
completeness, accuracy and 
alignment with automated business 
rules 

-Policy on Data Management has 
been developed and is being 
finalized 

-A new contract was awarded to 
implement streamlined data 
management procedures and 
templates 

-EDFacts Data Governance Board 
(EDGB) continues to promote and 
support stewardship of data 

-New certification language was 
updated in CSPR 

-The EDGB committed meeting 
time to exchange best practices, 
including NCES data review and 
quality control practices 


rd Felayarsxe Xen (0) ats) 


Identify efforts which 
could result in a more 
streamlined 
implementation of the 
tracking tool using key 
elements on the 2016-17 
CSPR to improve 
monitoring and 
correction of data quality 
issues 

-Finalize data 
management strategy 
and continuation with 
principal offices 
stewarding data quality, 
as part of the EDFacts 
Data Governance Board 
-Adopt new standard 
certification language in 
each data collection 
system; require states to 
provide additional 
justification for 
questionable data 
-Identify promising 
practices in monitoring 
grantee controls over 
data accuracy and share 
across principal offices 
-Leverage Single Audits 
to help assess grantee 
data quality 


Performance Goals, 
Tate tCers\ ce) comers tale 
Milestones 


Goal: Strengthen data 
management practices 
to support improved 
quality and efficient 
reporting to better 
enable evidence- 
based decision-making 
-Indicators: # of 
procedures or 
templates reviewed by 
the Department's Data 
Strategy Team for use 
across program office 
collections; # of data 
use sessions with 
stakeholders 

-Key milestones: 
Continue use of data 
tracking tool; identify 
data management 
framework used to 
identify procedures 
and templates for use 
across data life cycle 
of active collections; 
disseminate promising 
practices 


In FY 2017, the Department continued to make progress improving Information Technology 
System Development and Implementation, and the Office of Inspector General removed this 
issue from the list of FY 2018 Management Challenges. The Department has begun the 
Maturity Phase of the FITARA plan and, at the time of this report, the Department has 
completed seven of the seventeen maturity tasks. The continued improvements have been 
made evident on the most recent Biannual FITARA Scorecard in which the Department’s score 
increased a full letter grade, from a C+ to a B+. In an effort to continue improvement and gain 
efficiencies in IT spend, the Department has completed an “As-ls” visualization map of all IT 
systems and services. This effort will enable the Department to rationalize the IT portfolio by 
eliminating redundant IT services and consolidate existing services when possible. 
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Appendix A: FY 2017 Performance Metrics 

Appendix B: Data Validity and Verification for FY 2017 Annual Performance Report 

Appendix C: Data Validity and Verification for FY 2019 Performance Metrics 

Appendix D: Programs by Goal 

Appendix E: Summary of Performance Evaluations Conducted During FY 2017 and Expected 
During FY 2018 and FY 2019 

Appendix F: Glossary of Acronyms and Abbreviations 


Appendix A: FY 2017 Performance Metrics? 


GOAL 1: POSTSECONDARY EDUCATION, CAREER AND TECHNICAL EDUCATION, 
AND ADULT EDUCATION. Increase college access, affordability, quality, and 
completion by improving postsecondary education and lifelong learning opportunities 
for youths and adults. 


1.1.A Federal student loan delinquency rate WB DECREASE 
Historical ACTUALS TREND LINE (ACTUALS) 
Z 2014 | 2015 | 2016 | 2017 2017 2017 500% 
25.0% 
FY FY FY FY FY 20.0% ei 
2013 2014 2015 2016 2016 iv) 130m aa 
10.0% 
24.0% 21.7% 19.4% 18.6% 18.8% MET saps 
sms 2014-2015. 2016 2017 
Data Sources and Frequency of Collection: FSA Data Center; Annually. 
1.1.B Number of first-time FAFSA filers among high school seniors | 4 INCREASE 


Historical ACTUALS TREND LINE (ACTUALS) 


2014 2015 2016 2017 2017 2017 2,200,000 
2,150,000 ACTUAL 
2,100,000 2,148,070 
AY AY AY AY AY iv) 2060000 
2014-15 2015-16 2016-17 2017-18 2017-18 eege Pee ee oe Bee 
2,000,000 
2,021,691 2,009,155 | 2,015,138 2,148,070|2,016,304| MET Te2OOD 
e 2014 2015 2016 2017 


Data Sources and Frequency of Collection: FSA's Central Processing System (CPS); Annually. 


2 These performance metrics were established in relation to the FY 2014-18 Strategic Plan and 
their FY 2017 performance targets were included in the FY 2016 Annual Performance Report 
and FY 2018 Annual Performance Plan. Now that the Department has established the FY 2018- 
22 Strategic Plan, this Appendix represents the final report on these performance metrics and 
the prior Strategic Plan. 


APPENDICES 


1.1.C Index of national aggregate annual earnings of Vocation 
Rehabilitation (VR) consumers (based on the number of 
competitive integrated employment outcomes, hours 
worked, and hourly wages of VR consumers) 


Historical ACTUALS TREND LINE (ACTUALS) 


4D INCREASE 


1.1.C The metric is TBD in this report due to data not being available until 2018. 


Data Sources and Frequency of Collection: Rehabilitation Services Administration (RSA)-911; Annually. 


1.1.D Index of national aggregate earnings of Transition-Age Youth 
(based on the number of competitive integrated 
employment outcomes, hours works, and hours wages of 
VR Transition-Age Youth) 


Historical ACTUALS 


AB INCREASE 


2014 2015 2016 2017 
FY FY FY FY 

2014 2015 2016 2017 

$18,540,576 | $19,579,074 | $19,980,956 TBD 


2017 


LAV 
2017 


$19,866,354 


TBD 


TREND LINE (ACTUALS) 


25,000,000 


20,000,000 


15,000,000 


10,000,000 


2014 2015 2016 2017 2017 2017 | 
$80,000,000 I 2017 TARGET 
$60,000,000 
Ey Ey, FY FY FY Sa O0600 
2014 2015 2016 2017 2017 @ bial 
$20,000,000 
$61,800,214 | $64,688,320 | $66,791,933 | TBD $66,921,074 TBD 
2014 2015 2016 


2017 TARGET 


te) 


2014 2015 2016 


1.1.D The metric is TBD in this report due to data not being available until 2018. 


Data Sources and Frequency of Collection: Rehabilitation Services Administration-91 1; Annually. 


1.1.E Number of data points or other information reports released 


on the FSA Data Center AB increase 


Historical ACTUALS 


TREND LINE (ACTUALS) 


2014 2015 2016 2017 2017 2017 
20 
ACTUAL 
ae 1S... JOUZTARGET 
FY FY FY FY ‘v) 
2015 2016 2017 2017 10 
5 
N/A 12 15 16 15 MET 6 
2015 2016 2017 


1.1.E This metric was aligned with an FY 2016-17 APG. 


Data Sources and Frequency of Collection: FSA Data Center; Quarterly. 
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1.2.A Pell enrollment at IHEs with high graduation rates 4B INCREASE 
Historical ACTUALS TREND LINE (ACTUALS) 
2014 2015 2016 2017 2017 2017 _— 
25.0% 2017 TARGET 
AY AY AY AY one 
2012-13 2013-14 2014-15 2014-15 nae 
10.0% 
N/A 24.1% 24.5% 24.4% 26.0% | NOT MET = 
0.0% 
2015 2016 2017 


1.2.A_ Inprior year reports, the Department reported that the baseline data represented AY 2013-14. However, the baseline of 24.1% aligns with the 
AY 2012-13. The data table in this report is updated to accurately reflect all years. 


Data Sources and Frequency of Collection: Data from College Scorecard; Annually. 


1.2.B Number of states that develop or strengthen career 
pathways policies, guidance, or legislation 


Historical ACTUALS TREND LINE (ACTUALS) 


B® INCREASE 


2014 2015 | 2016 2017 2017 2017 
t ACTUAL 
| 40 es ase 20) TARGET 
FY FY FY FY rv) a9 
2015 2016 2017 2017 20 
10 
N/A 8 20 42 37 MET , 
2015 2016 2017 


Data Sources and Frequency of Collection: Development data from the National Skills Coalition, possibly supplemented with data from the Association 
of State Legislators and Center for Law and Social Policy; Annually. 


1.3.A Degree attainment among 25-34 year old age cohort 4B INCREASE 
Historical ACTUALS TREND LINE (ACTUALS) 
2014 2015 2016 2017 2017 2017 48% fit yiinpp 2017 TARGET, 
| | i ACTUAL 

Current 47% + 475% 
; 46% 
ae CPS Year CPS Year CPSYear | CPS Year x) oe 

2016 2016 
Year 2013 2014 2015 aie 
44.8% 45.7% 46.5% 47.5% | 48.4% |NOTMET) 7” 
2014 2015 2016 =. 2017 


1.3.A This metric was aligned with an FY 2016-17 Agency Priority Goal. 


Data Sources and Frequency of Collection: National Center for Education Statistics Digest of Education Statistics; Annually. 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 


APPENDICES 


1.3.B Persistence among first-time filing aid recipients 


Historical ACTUALS TREND LINE (ACTUALS) 


2014 2015 2016 | 
FY FY FY 
2014 2015 2016 
79.6% 79.5% 79.7% 


2017 


EY; 
2017 


82.6% 


4B INCREASE 


2017 


FY 
2017 


78.7%-80.7% 


2017 


v) 


MET 


84.0% 


82.0% 


80.0% 


78.0% 


76.0% 


ACTUAL 
82.6% 


MAXIMUM 
2017 TARGET 


errr Perret Pee «Set eis) 


2016 


2014 2015 2017 


Data Sources and Frequency of Collection: FSA’s Common Origination and Disburs 


ement (COD) system; Anually. 


Historical ACTUALS 


2014 | 2015 2016 
| 
AY 
mt 12 iin 13 2013-14 
556,696 573,911 578,583 
| 


1.4.A Number of STEM post-secondary credentials awarded 


2017 


AY 
2014-15 


627,399 


4B INCREASE 


2017 


AY 
2014-15 


637,672 


2017 


x) 


NOT MET 


650,000 
630,000 
610,000 
590,000 
570,000 
550,000 


TREND LINE (ACTUALS) 


2017 TARGET 


ACTUAL 
627,399 


2014 2015 2017 


2016 


Data Sources and Frequency of Collection: Integrated Postsecondary Education Data System (IPEDS) Data Center; Annually 


GOAL 2: ELEMENTARY AND SECONDARY EDUCATION. Improve the elementary and 
secondary education system’s ability to consistently deliver excellent instruction 
aligned with rigorous academic standards while providing effective support services to 
close achievement and opportunity gaps, and ensure all students graduate high school 


college- and career-ready. 


2.1.A Number of states/territories that have adopted 
college- and career-ready standards 


Historical ACTUALS 


2014 2015 2016 
sY sY sY 
2013-14 2014-15 2015-16 
51 51 51 


Data Sources and Frequency of Collection: N/A 


2017 


SY 
2016-17 


N/A 


4B INCREASE 


2017 


SY 
2016-17 


by. 


2017 


N/A 


TREND LINE (ACTUALS) 


2017 TARGET 


2014 


2015 


2016 


2.1.A With the passage of the Every Student Succeeds Act (ESSA), the Department no longer collects these data. 
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2.1.B Number of states/territories that are implementing 
next-generation reading and mathematics assessments, t INCREASE 
aligned with college- and career-ready standards 
Historical ACTUALS TREND LINE (ACTUALS) 
2014 2015 2016 2017 2017 2017 60 
- ~ —— = 2002 TARGET 
50 
40 
SY SY. SY; SY SY OC 30 
2013-14 2014-15 2015-16 2016-17 2016-17 a 
0 49 47 N/A 52 N/A bi 
0 
2014 2015 2016 
2.1.B-1 With the passage of the ESSA, the Department no longer collects these data. 
2.1.B-2 This metric was aligned with an FY 2016-17 APG. 
Data Sources and Frequency of Collection: N/A 
2.3.A Disparity in the rates of out-of-school suspensions for 
students with disabilities and youth of color @H DECREASE 
(youth of color metric) 
Historical ACTUALS TREND LINE (ACTUALS) 
2014 2015 2016 2017 2017 2017 12.0% 
[ 10.0% ° 
8.0% 
SY Si © 6.0% 
2013-14 2015-16 40% 
10.6% N/A TBD N/A N/A N/A —_ 
int di 5 0.0% 
i el : 2014 2016 
2.3.A The 2015-16 CRDC data for the 2016 actual will not be available until Spring 2018. 
Data Sources and Frequency of Collection: Civil Rights Data Collection (CRDC); Biennially 
2.3.B Disparity in the rates of out-of-school suspensions for 
students with disabilities and youth of color DECREASE 
(SWD, IDEA only metric) 
Historical ACTUALS TREND LINE (ACTUALS) 
2014 2015 2016 2017 2017 2017 
8.0% | | 
6.0% ° 
SY SY @ 
2013-14 2015-16 —_ 
2.0% 
6.6% N/A TBD N/A N/A N/A 
point disparity 0.0% 
2014 2016 


2.3.B The 2015-16 CRDC data for the 2016 actual will not be available until Spring 2018. 


Data Sources and Frequency of Collection: Civil Rights Data Collection; Biennially 
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2.4.A Number of persistently low graduation rate high schools B DECREASE 


Historical ACTUALS TREND LINE (ACTUALS) 


2014 2015 2016 2017 2017 2017 800 
sadennadannase anni aman see ene 
600 
sY sy SY SY aa 
2012-13 2013-14 2014-15 2015-16 
200 


737 680 605 N/A 630 N/A : 


2014 2015 2016 


2.4.A-1 With the passage of the ESSA, the Department no longer collects these data. 
2.4.A-2 This metric was aligned with an FY 2016-17 APG. 


Data Sources and Frequency of Collection: N/A 


2.4.B Percentage of SIG schools in Cohort 5 that are above the 
25th percentile in mathematics, as measured by their state | 4 INCREASE 
assessments 
Historical ACTUALS TREND LINE (ACTUALS) 
2014 2015 2016 2017 2017 2017 25.0% 
satemeneewerep nes iemeterusees 2017 TARGET 
20.0% 
SY SY, SY O 15.0% 
2013-14 2014-15 2015-16 10.0% 
5.0% 
N/A 19.7% 21.0% N/A 23.3% N/A 
0.0% 
2015 2016 


2.4.B With the passage of the ESSA, the Department no longer collects these data. 


Data Sources and Frequency of Collection: N/A 


2.4.C Percentage of SIG schools in Cohort 5 that are above the 
25th percentile in reading language arts, as measured by B® INCREASE 
their state assessments 


Historical ACTUALS 


TREND LINE (ACTUALS) 


2017 TARGET 


N/A 20.1% 


2.4.C With the passage of the ESSA, the Department no longer collects these data. 
Data Sources and Frequency of Collection: N/A 
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Historical ACTUALS 


2.5.A Number of public high school students who have taken at 
least one AP STEM exam 


4 INCREASE 


SY 
2012-13 


527,001 


Sy 
2013-14 


555,119 


SY 
2014-15 


592,410 


SY 


2015-16 


622,553 


SF 
2015-16 


691,541 


NOT MET 


TREND LINE (ACTUALS) 


750,000 
700,000 
650,000 
600,000 
550,000 
500,000 


2017 TARGET 


ACTUAL 
622,553 


2014 2015 2016 2017 


2.5.A The 2016 actual reported in this document is corrected from the FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan. 


Data Sources and Frequency of Collection: College Board/Advanced Placement (AP) administrative records; Annually 


GOAL 3: EARLY LEARNING. Improve the health, social-emotional, and cognitive 
outcomes for all children from birth through 3rd grade, so that all children, particularly 
those with high needs, are on track for graduating from high school college- and career- 


ready. 


Historical ACTUALS 


3.1.A Percentage of 4-year olds enrolled in state preschool 
programs 


4B INCREASE 


2014 2015 2016 2017 2017 2017 
SY sY sY 

2015-16 2016-17 2016-17 Lv) 

N/A N/A 85.5% 100% 92.7% MET 


TREND LINE (ACTUALS) 


ACTUAL 
100% 
2017 TARGET 


100% 
80% 
60% 
40% 
20% 

0.0% 


2016 


2017 


3.1.A-1 This metric was aligned with an FY 2016-17 APG. 
3.1.A-2 Some states received additional state funding for preschool allowing states to expand their communities and to reach more children, thus 
increasing the overall communities served from 230 to 250. Examples of states with increased funding are Alabama, New Jersey, New York and Tennessee. 


Data Sources and Frequency of Collection: Preschool Development Grants Annual Performance Reports, Data Workbook, Table A(1)(a) and (b); Annually. 


3.1.B Number of states with high-quality preschool programs 
standards 


Historical ACTUALS TREND LINE (ACTUALS) 


BP INCREASE 


ACTUAL 
18 


20 


i i i a a ee a ik 017 TARGET, 


2014 2015 2016 2017 


2014 2015 2016 | 2017 2017 2017 
SY sy SY SY SY iv) 
2013-14 2014-15 2015-16 2016-17 2016-17 
8 8 16 18 12 MET 
3.1.B-1 In 2016 this metric moved from using NIEER data to the Preschool Development Grants Annual Performance Reports. 


3.1.B-2 This metric was aligned with an FY 2016-17 APG. 


Data Sources and Frequency of Collection: Preschool Development Grants Annual Performance Reports, Data Workbook, Table 8; Annually. 
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3.2.A Number of states that require a teacher with a bachelor's 
degree in a state preschool program 


Historical ACTUALS TREND LINE (ACTUALS) 


4B INCREASE 


2014 2015 2016 2017 2017 2017 
SY sY sY SY sY (v) 
2013-14 2014-15 2015-16 2016-17 2016-17 
11 11 16 18 18 MET 


2014 


2015 


2016 2017 


3.2.A_ In 2016 this metric moved from using NIEER data to the Preschool Development Grants Annual Performance Reports. 


Data Sources and Frequency of Collection: Preschool Development Grants Annual Performance Reports, Data Workbook, Table 8; Annually. 


3.3.A Number of states collecting and reporting disaggregated 
data on the status of children at kindergarten entry using a 
common measure 


Historical ACTUALS TREND LINE (ACTUALS) 


4B INCREASE 


2014 2015 2016 2017 2017 2017 
sY sY SY sY sY ‘v) 
2013-14 2014-15 2015-16 2016-17 2016-17 
5 11 17 17 16 MET 


2014 


2015 


2016 2017 


Data Sources and Frequency of Collection: EDFacts; Annually 


GOAL 4: EQUITY. Increase educational opportunities for underserved students and 
reduce discrimination so that all students are well-positioned to succeed. 


2014 


SY 
2012-13 


81.4% 


2015 


SY. 
2013-14 


82.3% 


4.1.A National high school graduation rate 


Historical ACTUALS TREND LINE (ACTUALS) 


2016 


SY 
2014-15 


83.2% 


2017 


SY 
2015-16 


84.1% 


4B INCREASE 


2017 


SY 
2015-16 


85.0% 


2017 


x) 


NOT MET 


86.0% 
85.0% 
84.0% 
83.0% 
82.0% 
81.0% 
80.0% 
79.0% 


2017 TARGET 


ACTUAL 
84.1% 


2014 


2015 


2016 2017 


4.1.A_ This metric was aligned with an FY 2016-17 APG. 


Data Sources and Frequency of Collection: EDFacts, Data Groups 695 and 696; Annually 
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4.1.B Gap in the graduation rate between all students and 
students from low-income families 


Historical ACTUALS TREND LINE (ACTUALS) 


@H DECREASE 


2014 2015 2016 2017 2017 2017 = 
8.0% 
SY SY SY SY Gy || | RR allah eta pete 
2012-13 2013-14 2014-15 2015-16 | 2015-16 iv) 1M eh 
6.0% 
8.1% 7.7% 7.1% 65% | 7.4% MET |. 
. 2014 2015 2016 2017 


Data Sources and Frequency of Collection: EDFacts, Data Groups 695 and 696; Annually 


4.1.C Number of schools that do not have a gap or that decreased 
the gap between students from low-income families and 
the state average for all students 


AB INCREASE 


Historical ACTUALS TREND LINE (ACTUALS) 


2014 2015 2016 2017 2017 2017 
15000 [ = 
SY SY SY SY ‘v) 
2013-14 2014-15 2015-16 2015-16 
N/A 13,048 13,158 | 13,785 | 13,442 MET 


4.1.C This metric was aligned with an FY 2016-17 APG. 
Data Sources and Frequency of Collection: EDFacts, Data Groups 695 and 696; Annually. 


4.2.A Average number of cases substantively 
resolved, per investigative staff member 


Historical ACTUALS TREND LINE (ACTUALS) 


4 INCREASE 


2014 2015 2016 2017 2017 2017 
6.00 

Serer ere Tee aad 
FY FY FY x) 5,00 

2016 2017 2017 
4.00 
N/A N/A 5.17% 5.24% 5.27% | NOT MET 
3.00 
2016 2017 


Data Sources and Frequency of Collection: Office for Civil Rights Case Management System (CMS) and human resources dashboard; Annually. 
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4.2.B Number of technical assistance presentations and press releases 
on Office for Civil Rights’ work 


4B INCREASE 


Historical ACTUALS TREND LINE (ACTUALS) 


2014 2015 2016 2017 2017, ==—s-2017 iis 
FY FY FY x) 
2016 2017 2017 
N/A N/A 317 201 333 NOT MET 
2016 2017 


Data Sources and Frequency of Collection: Office for CMS and website; Quarterly. 


GOAL 5: CONTINUOUS IMPROVEMENT OF THE U.S. EDUCATION SYSTEM. Enhance 
the education system’s ability to continuously improve through better and more 
widespread use of data, research and evaluation, evidence, transparency, innovation, 


and technology. 


5.1.A Number of public data sets included in ED Data Inventory and thus 


INCREASE 
linked to Data.gov or ED.gov websites t 


TREND LINE (ACTUALS) 


Historical ACTUALS 


2014 2015 2016 2017 2017 2017 56 —_— 
FY FY FY FY FY rv) 
2014 2015 2016 2017 2017 
66 79 94 104 104 MET 
2014 2015 2016 2017 


Data Sources and Frequency of Collection: Data Strategy Team Data Inventory and the public Department's Data Inventory at 
http://datainventory.ed.gov; Quarterly. 


5.1.B Number of states linking K-12 and postsecondary data with 


INCREASE 
workforce data t 


Historical ACTUALS TREND LINE (ACTUALS) 


2014 2015 2016 2017 2017 2017 
FY FY FY FY FY rv) 
2014 2015 2016 2017 2017 
20 24 28 28 25 MET 


2014 


oreseeueene 


2016 


2017 


2015 


Data Sources and Frequency of Collection: Statewide Longitudinal Data Systems (SLDS) grant monitoring (monthly updates from states, annual 
performance reports, final performance reports, and site visits); Quarterly. 
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Historical ACTUALS 


5.1.C Number of states actively using data systems to 
support and inform improvements 


4 INCREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY FY FY ‘v) 

2014 2015 2016 2017 2017 

18 28 32 34 26 MET 


TREND LINE (ACTUALS) 


2014 


2017 


Data Sources and Frequency of Collection: State Longitudinal Data Systems (SLDS) grant monitoring (monthly updates from states, annual 
performance reports, final performance reports, and site visits); Quarterly. 


N/A 


Historical ACTUALS 


2015 


Br 
2015 


91,011 


5.1.D Web traffic to the College Scorecard 


2016 


FY 
2016 


1,427,797 


964,982 


4B INCREASE 


FY 
2017 


| 1,800,000 


2017 


2017 


x) 


NOT MET 


TREND LINE (ACTUALS) 


EY17 TARGET 


2015 


ACTUAL 
964,982 


2017 


Data Sources and Frequency of Collection: Google Analytics data from College Scorecard; Quarterly. 


5.2.A Average time to close “cases” (PTAC + FPCO) 


Historical ACTUALS TREND LINE (ACTUALS) 


@ DECREASE 


2014 2015 2016 2017 2017 2017 

Ey FY FY FY EY iv) 
2014 2015 2016 2017 2017 

9.00 days 4.90 days 6.06 days | 4.57 days | 6.71 days MET 


2014 


2017 TARGET 


ACTUAL 
457 


2017 


Data Sources and Frequency of Collection: Case Tracking System (CTS) Monthly Metric Reports; Quarterly. 


5.3.A Percentage of select new (non-continuation) discretionary grant 
dollars that reward evidence 


Historical ACTUALS TREND LINE (ACTUALS) 


BP INCREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY FY FY 

2014 2015 2016 2017 2017 iv) 

15.9% 29.4% 29.9% 30.03% 20% MET 


40.0% 
30.0% 
20 
10.0% 


0.0% 


2014 


| _ 2017 TARGET 


eee eee 


ACTUAL 
303% 


2017 


5.3.A This metric was aligned with an FY 2016-17 APG. 


Data Sources and Frequency of Collection: Forecast Report issued by the Office of the Chief Financial Officer (OCFO) and final Funding Reports from 
relevant programs; Annually. 
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Historical ACTUALS 


5.3.B Number of peer-reviewed, full-text resources in the Education 
Resources Information Center (ERIC) 


4B INCREASE 


2014 2015 2016 | 2017 2017 2017 
FY FY FY FY FY 

2014 2015 2016 2017 2017 iv) 

27,292 36,197 47,573 60,009 40,892 MET 


TREND LINE (ACTUALS) 


ACTUAL 
60,009 


2017,TARGET. 


2014 2015 2016 2017 


Data Sources and Frequency of Collection: Education Resources Information Center (ERIC); Quarterly. 


5.3.C Number of visits to the What Works Clearinghouse (WWC) website 


Historical ACTUALS TREND LINE (ACTUALS) 


BD INCREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY FY 

2015 2016 2017 2017 iv) 

N/A 1,822,000 3,756,724 4,418,125 2,164,536 MET 


ACTUAL 
4,418,125 


2017 TARGET 


2015 


2016 2017 


Data Sources and Frequency of Collection: What Works Clearinghouse (WWC) website analytics provided monthly by the 
WWC website contractor; Quarterly. 


Historical ACTUALS 


5.3.D Number of completed project evaluations from grantees of 
select discretionary grant programs in a given fiscal year that meet t INCREASE 
What Works Clearinghouse (WWC) Evidence Standards 


2014 2015 2016 2017 2017 2017 
FY FY FY FY x) 
2015 2016 2017 2017 
N/A 2 20 4 10 NOT MET 


TREND LINE (ACTUALS) 


2017 TARGET 


ACTUAL 
4 


2014 


2015 2016 2017 


5.3.D This metric was aligned with an FY 2016-17 APG. 


Data Sources and Frequency of Collection: Discretionary grant slate memoranda, discretionary grant financial forecasts and reports from OCFO, 
and the What Works Clearinghouse; Quarterly. 
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5.4.A Percentage of schools in the country that have actual internet 
bandwidth speeds of at least 100 Mbps 


Historical ACTUALS TREND LINE (ACTUALS) 


B® INCREASE 


2014 2015 2016 2017 2017 2017 
100% ACTUAL 
88% 017 TARGI 
80% cena SIR TRCET, 
SY SY SY sy SY @ 60% 
2014-15 2015-16 2016-17 2017-18 2017-18 40% 
20% 
N/A 59% 78% 88% 80% MET 
0% 
2015 2016 2017 


5.4.A_|n the FY 2016 Annual Performance Report and FY 2018 Annual Performance Plan, the Department used data from the Consortium for School 
Networking for this metric. This report includes data from the Education Superhighway (ESH); as such, the actuals for past performance have been 
updated. Baseline data collected by ESH directly from districts during the 2013-14 school year showed that 37% of schools in the country had actual 


Internet bandwidth speeds of at least 100 Mbps. This data has been published by ESH in its 2015, 2016, and 2017 “State of the States” reports. 


Data Sources and Frequency of Collection: Education Superhighway; Annually. 


GOAL 6: U.S. DEPARTMENT OF EDUCATION CAPACITY. Improve the organizational 
Capacities of the Department to implement this strategic plan. 


Historical ACTUALS 


6.1.A Percentage of selections made per job opportunity 
announcement (JOA) 


4B INCREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY FY 

2015 2016 2017 2017 iv) 

N/A 46.4% 70% 110% 51.2% MET 


TREND LINE (ACTUALS) 


120.00% 
100.00% 


ACTUAL 
110% 


80.00% 
60.00% 
40.00% 
20.00% 


2017 TARGET 


0.00% 


2014 


2015 2016 


2017 


6.1.A Multiple selections were made from some JOAs and resulted in current year actuals exceeding 100%. 


Data Sources and Frequency of Collection: EDHires (Monster's electronic hiring management system); Annually. 


2014 


67% 


6.1.B EVS Employee Engagement Index 


Historical ACTUALS 


B® INCREASE 


NOT MET 


TREND LINE (ACTUALS) 


74.00% 
72.00% 
70.00% 


2017 TARGET 


68.00% 


66.00% 


64.00% 


2014 


2015 2016 


Data Sources and Frequency of Collection: OPM Federal Employee Viewpoint Survey; Annually. 
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6.1.C Time to hire 


Historical ACTUALS 


4B INCREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY FY FY 

2014 2015 2016 2017 2017 iv) 

85% 67.6% 79.1% 87.6% 70% MET 


100.00% 
80.00% 
60.00% 
40.00% 
20,00% 

0.00% 


TREND LINE (ACTUALS) 


ACTUAL 
87.6% 
2017 TARGET 


2014 2015 2016 2017 


6.1.C This metric represents the percentage of hiring actions completed within 90 days (i.e., 87.6% of FY17 hires were completed within 90 days). 


Data Sources and Frequency of Collection: Federal Personnel/Payroll System (FPPS) Datamart; Quarterly. 


2014 


FY 
2014 


50% 


6.1.D Effective Communication Index 


Historical ACTUALS 


2015 


FY 
2015 


51% 


2016 


EY 
2016 


50% 


2017 


AB INCREASE 


2017 


FY 
2017 


52% 


2017 


x) 


NOT MET 


53.00% 


52.00% 


51.00% 


50.00% 


49.00% 


TREND LINE (ACTUALS) 


2017 TARGET 


2014 2015 


2016 2017 


Data Sources and Frequency of Collection: OPM Federal Employee Viewpoint Survey; Annually. 


6.2.A Percentage of Department grant recipients without any Single 
Audit Findings 


Historical ACTUALS TREND LINE (ACTUALS) 


4B INCREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY 

2016 2017 2017 iv) 

N/A N/A 86.1% 87.6% 86.1% MET 


88.0% 
87.5% 


ACTUAL 
87.6% 


87.0% 
86.5% 
86.0% 
85.5% 


2017 TARGET 


85.0.% 


2016 


2017 


Data Sources and Frequency of Collection: Federal Audit Clearing 


house (FAC); Quarterly. 


Historical ACTUALS 


2014 2015 2016 
FY FY FY 

2014 2015 2016 

97% 98.0% 95.8% 


6.2.B Compliance rate of contractor evaluation performance reports 


4B INCREASE 


FY 
2017 


100% 


NOT MET 


95.00% 


90.00% 


85.00% 


TREND LINE (ACTUALS) 


2017 TARGET 


2014 2015 


2016 


Data Sources and Frequency of Collection: Past Performance Information Retrieval System (PPIRS) Compliance Metric Report; Quarterly. 
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ICES 


Historical ACTUALS 


2014 2015 2016 
FY FY 
2015 2016 
N/A 7.73 7.58 
| 
1 


6.3.A Overall average impact score of the Department's technical 
assistance in helping build state capacity to implement 
education reforms 


2017 


FY 
2017 


5.82 


4B INCREASE 


2017 


FY 
2017 


7.75 


2017 


x) 


NOT MET 


Data Sources and Frequency of Collection: Annual Grantee Satisfaction Survey; Annually. 


TREND LINE (ACTUALS) 


2017 TARGET 


2015 


2016 


2017 


6.4.A Number of EDIT security incidents 


Historical ACTUALS TREND LINE (ACTUALS) 


@H DECREASE 


2014 2015 2016 | 2017 2017 2017 
FY FY FY FY FY 

2014 2015 2016 2017 2017 iv) 

445 580 291 187 291 MET 


800 


2017 TARGET 


2014 


2015 2016 


2017 


Data Sources and Frequency of Collection: Cyber Security Assessment and Management (CSAM) and RSA Security Operations management 
(SecOps); Quarterly. 


Historical ACTUALS 


6.4.B EVS Results Oriented Performance Culture Index 


4B INCREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY FY FY 

2014 2015 2016 2017 2017 x) 

56% 57% 56% 57% 58% NOT MET 


59.00% 


58.00% 


57.00% 


56.00% 


55.00% 


TREND LINE (ACTUALS) 


2017 TARGET 


2014 


2015 2016 


Data Sources and Frequency of Collection: OPM Federal Employee Viewpoint Survey; Annually. 


6.4.C EVS Leadership and Knowledge Management Index 


Historical ACTUALS TREND LINE (ACTUALS) 


AB INCREASE 


2014 2015 2016 2017, | 2017 | 2017 
| | 
FY FY FY FY FY 
2014 2015 2016 2017 2017 x] 
61% 62% 61% 61% 64% NOT MET 


65.00% 
64.00% 
63.00% 
62.00% 
61.00% 
60.00% 
59.00% 


2017 TARGET 


2014 2015 2016 


2017 


Data Sources and Frequency of Collection: OPM Federal Employee Viewpoint Survey; Annually. 
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6.4.D Total usable square footage 


Historical ACTUALS 


@& DECREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY FY FY 
2014 2015 2016 2017 2017 x] 
1,533,239 1,530,864 1,367,000 1,382,553 1,381,775 NOT MET 


TREND LINE (ACTUALS) 


ACTUAL 
1,382,553 
2017 TARGET 


1,200,000 


1,500,000 


1,000,000 


500,000 


0 


2014 


2015 2016 2017 


Data Sources and Frequency of Collection: Department's Master Space Management Plan; Annually. 


6.4.E Rent cost 


Historical ACTUALS 


@ DECREASE 


2014 2015 2016 2017 2017 2017 
FY FY FY FE FY iv) 
2014 2015 2016 2017 2017 
$74,100,000 $72,720,099 $72,149,828 | $65,527,456 $74,470,439 MET 
! | 


TREND LINE (ACTUALS) 


80,000,000 


2017 TARGET 


nilamnician wea wwe minaminpminee merce 


75,000,000 +-—- 


70,000,000 


ACTUAL 
65,527,456 


65,000,000 


60,000,000 


2014 


2015 2016 2017 


Data Sources and Frequency of Collection: Department's Master Space Management Plan; Quarterly. 


N/A = Not applicable 


TBD = To be determined 


Academic Year (AY) is a collegiate year spanning August-May; School Year (SY) spans August- 
July and is aligned with a P-12 school year; Fiscal Year (FY) corresponds to a federal fiscal 
year; Calendar Year (CY) spans January-December. 


Note on performance metrics and targets: These metrics were established as a part of the 
FY 2014-18 Strategic Plan. Metrics may be updated from previous reports or revised to reflect 
awareness of more accurate data or clarifications. Such updates or revisions are identified in 


footnotes. 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 


73 


APPENDICES 


Appendix B: Data Validity and Verification for FY 2017 
Annual Performance Report 


The Government Performance and Results Act Modernization Act of 2010 requires 
agencies to describe the accuracy and reliability of data presented. Details of how the 
Department assesses the completeness and reliability of the data reported for FY 
2017 are presented as part of this appendix, and known limitations of the data are also 
included. 


To the best of my knowledge, the data verification and validation process and the data 
sources used provide, to the extent practicable, complete, and reliable performance 
data pertaining to goals and objectives in our FY 2014-18 Strategic Plan. Through a 
process of continuous improvement, the Department regularly assesses its validation 
process and welcomes input from stakeholders. 


Joseph C. Conaty 
Senior Policy Advisor 


Office of the Deputy Secretary 
February 12, 2018 
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GOAL 1: POSTSECONDARY EDUCATION, CAREER AND TECHNICAL EDUCATION, AND 
ADULT EDUCATION. Increase college access, affordability, quality, and completion by 
improving postsecondary education and lifelong learning opportunities for youths and 


adults. 


Metric 
No. 


Performance 
Metric (Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


1.1.A 


Federal student 
loan delinquency 
rate 


Federal Student 
Aid (FSA) Data 
Center 


National Student Loan Data System (NSLDS) is currently the 
source of the data. Verification and validation occur in three 
aspects of the process: 

(1) Data providers (servicers in this case) transmit data to 
NSLDS where the data are subject to edits and screening. 
FSA works with the servicers to resolve anomalies. 

(2) Users who analyze the data sometimes identify 
anomalies in the course of their analytic roles and work with 
experts in NSLDS to resolve. In some cases, NSLDS will 
again work with the servicer(s) to resolve. 

(3) The report providing the data for this metric has been 
validated. This means that the programming and logic have 
been independently reviewed. 


1.1.B 


Number of first- 
time Free 
Application for 
Federal Student 
Aid (FAFSA) filers 
among high school 
seniors 


FSA's Central 
Processing 
System 


The data we use originate from FSA operational systems. 
These systems, as a part of their operations, have 
procedures in place to address potential data quality issues. 
These metrics report figures about data that our operational 
systems maintain, and we do not manipulate it in any way. If 
anomalies persist, they exist in the operational systems. 

We have a consistent and disciplined process in place for 
querying our systems for data. The process includes having 
a separate data analyst from a different office within FSA 
validate the accuracy of the query and the resulting data. 
This process also is used for anomalous data. 

In addition to the above controls, queries and calculations are 
simultaneously conducted on data from previous years to 
ensure technical definitions remain consistent and 
calculations by the business intelligence team as part of the 
validation. 

The Customer Analytics Group is responsible for the primary 
calculation of the metric using the data from FSA. Business 
Intelligence is responsible for the technical validation of the 
metric, which is done by reviewing for accuracy the query 
used to pull the data and all calculations made with the data. 
Finally, the Financial Reporting and Analysis Branch is 
responsible for ensuring that documentation is complete and 
archived. 

Since year of high school graduation is not asked on the 
FAFSA, several assumptions are made and specific criteria 
were used (such as age of applicant) to identify those likely 
to be high school seniors. These assumptions and criteria 
are applied consistently across all baseline years and future 
calculations. 

These calculations also restrict the application period to the 
first nine months of the application cycle (the close of the 
fiscal year), rather than the entire 18 months. Since most 
applicants, including high school seniors, file their FAFSA 
prior to the start of the upcoming academic year (usually 
before fiscal year end), this decision better aligns the 
performance metric with the fiscal year where most of the 
performance occurred. The alternative is waiting for the close 
of the 18-month cycle, where a performance metric would 


mostly reflect performance from an earlier fiscal year. 
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pitte es ionnante Data Source(s) Data Verification and Validation Overview 

oO. Metric (Indicator) 

1.1.C Index of national RSA-911 Each grantee’s RSA-911 is submitted as a text file to RSA 
aggregate annual via an SFTP server. We run an edit program that converts 
earnings of the text file into MS Access and verifies the accuracy of the 
Vocation data. Each of the 215 data elements is verified. If errors are 
Rehabilitation (VR) identified by the application, the grantee is requested to 
consumers (based correct the errors and resubmit the data. Once the data file is 
on the number of corrected and resubmitted, it is checked again using the 
competitive same process. If anomalies are identified, the grantee must 
integrated provide an explanation and verify that the data are correct. 
employment RSA provides the agencies with the edit program that we use 
outcomes, hours and are encouraged to run their data using the program 
worked, and hourly often, at least quarterly, during the year. In addition, agencies 
wages of VR have edits in their own systems which run additional checks. 
consumers) 

1.1.D Index of national 
aggregate 
earnings of 
Transition-Age 
Youth (based on 
the number of 
competitive 
integrated 
employment 
outcomes, hours 
works, and hours 
wages of VR 
Transition-Age 
Youth) 

1.1.E Number of data Federal Student The Validation Subject Matter Expert (SME) works with the 
points or other Aid (FSA) Data Request SME to understand the solution used to fulfill the 
information reports | Center request, checking all results, code, and documentation 
released on the produced by the Request SME. Additionally, the Validation 
FSA Data Center SME ensures there are no data disclosure issues that need 

to be addressed. In the event the Validation SME cannot 
validate the request results, he or she contacts the Request 
SME and the Data Request Team (DRT) to resolve any 
issues necessary to successfully complete the validation. 
The Validation SME completes his or her portion of the 
Request Template and notifies the DRT that the validation 
step is complete. Upon receipt of the data that was 
requested, the DRT confirms with the assigned Validation 
SME that the results have been validated and documented 
per their selected solution and enters it into the DRT 
database 

1.2.A Pell enrollment at Data from college Validation of the data is conducted by the National Center for 
IHEs with high Scorecard Education Statistics (NCES) through its annual IPEDS data 
graduation rates validation process. NCES will ensure accuracy, while 

OPEPD/ Policy and Program Studies Service (PPSS) will 
incorporate the data into the Scorecard datasets. 

Analysis will be conducted to identify anomalous data. They 
will be handled on a case-by-case basis, as most anomalies 
are due to variation from year-to-year or small n-size and are 
already pooled and suppressed before publishing the data. 
Other data problems may necessitate additional runs, 
qualifications, or suppression for stability reasons. 
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silts Js enenes Data Source(s) Data Verification and Validation Overview 
oO. Metric (Indicator) 

1.2.B Number of states Development data | The Department cannot independently verify the data, but 
that develop or from the National relies on the tracking of the National Skills Coalition. There 
strengthen career Skills Coalition, are no processes in place to check for anomalous data. 
pathways policies, | possibly 
guidance, and or supplemented with 
legislation data from the 

Association of 
State Legislators 
and Center for 
Law and Social 
Policy 

1.3.A Degree attainment | NCES Digest of Data available at 
among 25-34 year | Education https://nces.ed.gov/programs/digest/d16/tables/dt16 104.30. 
old age cohort Statistics, Table asp, Number of persons age 18 and over, by highest level of 

104.30 educational attainment, sex, race/ethnicity, and age: 2016. 
Tabulated from Current Population Survey data, U.S. 
Census. 
The underlying data is verified by Census prior to publication. 
Data quality and limitations are documented in: 
http://www.census.gov/prod/2006pubs/tp-66. pdf. 
NCES tabulates the data, which are verified prior to 
publication according to NCES guidelines. 

1.3.B Persistence FSA’s Common Data used for these calculations are based on counts from 
among firs-time Origination and operational systems (number of recipients and number of 
filing aid recipients | Distribution system | applicants). Moreover, standardized queries are used to 

rerun and match calculations for earlier cycles. Therefore, the 
metric as defined should be considered verified and valid. 
Data used for these calculations are based on counts from 
operational systems (number of recipients and number of 
applicants meeting certain criteria). Anomalous data would 
suggest there are recipients that are not recipients (or vice 
versa) or applicants that are not applicants (or vice versa). A 
variety of internal controls are in place tracking both of these 
processes. 
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Engineering and 
Math (STEM) post- 
secondary 
credentials 
awarded 


System (IPEDS) 
Data Center 


Doctor's Degrees: 


(324.25, 324.30 
and 324.35) 


Metric Performance ee pages F 
No. Metric (Indicator) Data Source(s) Data Verification and Validation Overview 
1.4.A Number of Integrated Doctor's Degrees: (324.25, 324.30 and 324.35) 
Science, Postsecondary https://nces.ed.gov/programs/digest/d16/tables/dt16_324.25.asp 
Technology, Education Data https://nces.ed.gov/programs/digest/d16/tables/dt16_324.30.asp 


https://nces.ed.gov/programs/digest/d16/tables/dt16_324.35.asp 
Master's Degrees: (323.30, 323.40, 323.50) 
https://nces.ed.gov/programs/digest/d16/tables/dt16_323.30.asp 


https://nces.ed.gov/programs/digest/d16/tables/dt16_323.40.asp 
https://nces.ed.gov/programs/digest/d16/tables/dt16_323.50.asp 


Bachelor's Degrees: (322.30, 322.40, 322.50) 
https://nces.ed.gov/programs/digest/d16/tables/dt16_322.30.asp 


https://nces.ed.gov/programs/digest/d16/tables/dt16_322.40.asp 
https://nces.ed.gov/programs/digest/d16/tables/dt16_322.50.asp 


Associate's Degrees: (321.30, 321.40, 321.50) 
https://nces.ed.gov/programs/digest/d16/tables/dt16_321.30.asp 
https://nces.ed.gov/programs/digest/d16/tables/dt16_321.40.asp 
https://nces.ed.gov/programs/digest/d16/tables/dt16_321.50.asp 
Certificates: (320.20) 
https://nces.ed.gov/programs/digest/d16/tables/dt16_320.20.asp 
IPEDS collects completions by Classification of Instructional 
Programs (CIP) Code which may be aggregated into STEM 
and non- STEM counts. The Department of Education does 
not currently have a single definition for which CIP codes are 
STEM. 

Data quality and limitations are identified in IPEDS First Look 
Publications, “Data Collection Procedures,” and IPEDS 
methodology available at 


https://nces.ed.gov/pubs2015/2015098.pdf. Data are 
checked by NCES for consistency. 
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GOAL 2: ELEMENTARY AND SECONDARY EDUCATION. Improve the elementary and 
secondary education system’s ability to consistently deliver excellent instruction aligned 
with rigorous academic standards while providing effective support services to close 
achievement and opportunity gaps, and ensure all students graduate high school 


college- and career-ready. 


Metric 
No. 


Performance 
Metric (Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


2.1.A 


Number of 
states/territories 
that have adopted 
college- and 
career-ready 
standards 


ESEA Flexibility 
Monitoring 


With the passage of the ESSA, the Department no longer 
collects these data. 


2.1.B 


Number of 
states/territories 
that are 
implementing next- 
generation reading 
and mathematics 
assessments, 
aligned with 
college- and 
career-ready 
standards 


ESEA Flexibility 
Monitoring 


With the passage of the ESSA, the Department no longer 
collects these data. 


2.3.A 


Disparity in the 
rates of out-of- 
school 
suspensions for 
students with 
disabilities and 
youth of color 
(youth of color 
metric) 


CRDC 


2.3.B 


Disparity in the 
rates of out-of- 
school 
suspensions for 
students with 
disabilities and 
youth of color 
(Students With 
Disabilities (SWD), 
IDEA only metric) 


CRDC 


OCR strives to ensure CRDC data are an accurate and 
comprehensive depiction of student access to educational 
opportunities in school districts. The CRDC submission 
system conducts automatic edit checks to validate the format 
of the data and the system also performs business rule 
validations. For suspensions, the business rule validations 
include comparisons of disaggregated data to totals and 
enrollment counts. Districts must address the quality issues 
identified before the accuracy of the data can be certified. 
Only the district superintendent, or the superintendent's 
designee, may certify the data submission. Ultimately, the 
quality of the CRDC data depends on accurate collection and 
reporting by the participating districts. 


24.A 


Number of 
persistently low 
graduation rate 
high schools 


EDFacts 


With the passage of the ESSA, the Department no longer 
collects these data. 


2.4.B 


Percentage of 
School 
Improvement 
Grant (SIG) 
schools in Cohort 
5 that are above 
the 25" percentile 
in mathematics, as 
measured by their 
state assessments 


Analytic dataset 
produced by the 
contractor for the 
SIG National 
Summary 


With the passage of the ESSA, the Department no longer 
collects these data. 
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high school 
students who have 
taken at least one 
AP STEM exam 


administrative 
records 


grrig per formance Data Source(s) Data Verification and Validation Overview 
oO. Metric (Indicator) 
2.4.C Percentage of SIG | Analytic dataset With the passage of the ESSA, the Department no longer 
schools in Cohort produced by the collects these data. 
5 that are above contractor for the 
the 25" percentile | SIG National 
in reading Summary 
language art, as 
measured by their 
state assessments 
2.5.A Number of public College Board/AP | The College Board does not collect or report race/ethnicity 


based on the federal guidelines. Examinees are asked to 
select one of the options noted in the data. The College 
Board Public School List is updated annually by state 
educational agencies; thus, small changes to the list over 
time are to be expected as schools open, close, and/or 
merge. Students are assigned to graduating cohorts based 
on self-reported information (i.e., grade level and/or 
graduation year) provided at the time of registration (in the 
case of SAT) or test administration (in the case of AP and 
PSAT). The College Board matches students’ data across 
programs to identify the most recent valid value when 
assigning students to cohorts. 
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GOAL 3: EARLY LEARNING. Improve the health, social-emotional, and cognitive outcomes 
for all children from birth through 3rd grade, so that all children, particularly those with 
high needs, are on track for graduating from high school college- and career-ready. 


Lira aed ookig Data Source(s) Data Verification and Validation Overview 
No. Metric (Indicator) 
3.1.A Percent of 4-year Preschool Annual Performance Report data are collected externally. The 
olds enrolled in Development data are collected primarily through a performance report 
state preschool Grants Annual template and a data workbook submitted by State Project 
programs Performance Directors for the grants. 
Reports, Data are collected by program offices through a technical 
Data Workbook, assistance contractor using a performance report template and 
Table A(1)(a) and self- administered electronic submission tool. The data 
(b) collection period for one year is reported in the following year’s 
3.1.B Number of states Preschool Annual Performance Report submission. For example, school 
with high—quality Development year 2015-2016 data is reported in February of 2016. The 
preschool Grants Annual Technical Assistance (TA) contractor performs an analysis and 
programs Performance data anomalies are clarified and verified with the State Project 
standards Reports, Director. 
3.2A Number of states Data Workbook, 
that require a Table 8 
teacher with a 
bachelor’s degree 
in a state preschool 
program 
3.3.A Number of states EDFacts The project officers and our TA provider work with the Race to 
collecting and the Top-Early Learning Challenge grantees to verify and 
reporting validate the data. The project officers and our TA provider work 
disaggregated data with the Race to the Top-Early Learning Challenge grantees to 
on the status of check for anomalous data. If anomalous data are discovered, 
children at project officers, the TA provider and the grantee meet and 
kindergarten entry discuss. 
using a common 
measure 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 


81 


APPENDICES 


GOAL 4: EQUITY. Increase educational opportunities for underserved students and reduce 
discrimination so that all students are well-positioned to succeed. 


that do not have a 
gap or that 
decreased the gap 
between students 
from low-income 
families and the 
state average for 
all students 


PerroLmanice Data Source(s) Data Verification and Validation Overview 

oO. Metric (Indicator) 

4.1.A National high EDFacts EDFacts works with the data stewards to determine the 
school graduation appropriate business rule checks for these data. The checks 
rate that were done on these data include: 

4.1.B Gap in the EDFacts File validation and format checks: Identify file submissions that 
graduation rate do not conform to the accepted file format, the data universe 
between all (school/LEA/state directory), or the reporting requirements 
students and (mandatory reporting fields). 
students from low- Submission Edit Business Rules: Submission edits and 
income families warnings ensure that the data meet or exceed an acceptable 

4.1.C Number of schools | EDFacts level of reasonability by checking the values entered in a field 


against other similar values in the same file or across files. If a 
discrepancy is found (i.e., a value falls outside of the 
acceptable range), a submission error or warning is issued. 
Unlike format and validation edits, submission edits and 
warnings are programmed into the EDFacts Submission 
System and applied to the data after they are in the staging 
database. 

Coordinated Data Quality Reviews (CDQR): Identify potential 
errors and anomalies related to completeness, consistency, 
and comparability in the file submissions that would affect the 
quality and usability of data in the files. 

CDQR Process: At the end of a collection period, EDFacts runs 
checks to validate data quality of submitted data and presents 
all results to program offices. It is the responsibility of the 
program offices, in consultation with the EDFacts staff, to 
determine which identified errors to escalate to states for 
further review. The EDFacts Partner Support Center (PSC) 
sends these errors to states for remediation, which take the 
form of explanations for data anomalies, data file submission 
updates, and data corrections. OESE should be considered the 
point-of-contact for identifying which errors were escalated and 
the result of those escalations. 

Path to public release and national rates: For each collection 
cycle states report their calculated cohort counts and ACGR 
graduation rates. These counts and rates are tested against a 
number of submission edit business rules and the Adjusted 
Cohort Graduation Rate (ACGR) Coordinated Data Quality 
Review for format, consistency, completeness, and 
comparability. Those rule checks are delivered to the OESE for 
follow-up correspondence through the EDFacts PSC with the 
states. The program office, in consultation with NCES EDFacts 
staff and other stakeholders, meet to determine which issues 
identified by the business rules should be raised with the state 
for explanation, update, or correction. Following that review, 
data are then aggregated to the national level for the purpose 
of calculating and publishing a national rate. NCES processes 
these data to force conformity of reporting categories (mapping 
reported “Major Racial/Ethnic Groups” to the traditional five 
racial/ethnic groups) and addresses any issues raised by 
missing data. After imputations are made, NCES produces a 
national rate for the country as a whole; a rate which is 
representative of every state. Once produced, NCES 
documents any remaining issues with these data, and the 
aggregation and imputation methodology in a public report. 
That report, documentation, and the associated data tables are 
put through several stages of review including independent 
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Metric 
No. 


Performance 
Metric (Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


reviews at the division (Administrative Data), center (NCES), 
and principal office (IES) levels prior to public dissemination. 
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investigative staff 
member 


System (CMS) 
system and human 
resources 
dashboard 


gee ier formance Data Source(s) Data Verification and Validation Overview 

oO. Metric (Indicator) 

42.A Average number of | Office for Civil OCR captures up-to-date information related to substantive 
cases substantively | Rights Case closures of cases through its Case and Activity Management 
resolved, per Management System (CAMS). The CAMS is a formal IT Investment, which 


includes a business case and its own performance metrics 
tracked through ED’s IT Investment Review process. The 
CAMS includes the Case Management System (CMS), Activity 
Management System, and Document/Record Management 
(DM/RM) components. In order to track this Metric, the staff in 
OCR’s 12 enforcement offices are responsible for updating and 
maintaining accurate case statuses in the CAMS’s CMS. This 
specific Metric data is extracted from the CAMS for reporting 
purposes by OCR’s IT Specialist who is also the CAMS Project 
Manager and technical expert. This staff person then delivers 
the data and analysis to the Goal 4 Strategic Goal Leader for 
reporting. 

The verification and validation of CAMS data utilize (1) rules 
built into the CAMS’s CMS component; (2) periodic checks on 
questionable data; (3) guidance and reference on data entries; 
and (4) regular training and guidance for primary users 
imputing the data. 

(1) Rules built in CMS: 

Samples of CMS validations: 

- Docket number is assigned by the CMS based on four 
required data (Active Office, TA Type, Nature/Type of 
Assignment, and Initial Contact/Target). 

- Entry date must be validated. 

(2) Periodic checks on questionable data: (quarterly) 

- Missing entries: such as TA completion without (a) date of 
final activity, (b) activity log including subject area, completion 
date and location. 

(3) Guidance and reference materials for use by primary users 
making data entries are posted at the OCR SharePoint site and 
are available to all OCR staff: 

- CMS Data Entries requirements. 

- CMS User Manual. 

(4) Users Training: 

User training is available as needed. OCR also provides 
training through a “peer helper” or “train the trainer” model for 
staff in the field offices and headquarters. 
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issuance of press 
releases, and 
issuance of 
resolution 
agreements on 
OCR work 


website 


gle ies formance Data Source(s) Data Verification and Validation Overview 

oO. Metric (Indicator) 

4.2.B Number of Office for Civil The verification and validation of CAMS data utilize (1) rules 
technical Rights Case built into the CAMS’s CMS component; (2) periodically checks 
assistance Management on questionable data; (3) guidance and reference on data 
presentations, System (CMS) and | entries; and (4) regular training and guidance for primary users 


inputting the data. 

(1) Rules built in CMS: 

Samples of CMS validations: 

- Docket number is assigned by the CMS based on four 
required data (Active Office, TA Type, Nature/Type of 
Assignment, and Initial Contact/Target). 

- Entry date must be validated. 

(2) Periodic checks on questionable data: (quarterly) 

- Missing entries: such as TA completion without (a) date of 
final activity, (b) activity log including subject area, completion 
date and location. 

3) Guidance and reference materials for use by primary users 
making data entries are posted at the OCR SharePoint site and 
are available to all OCR staff: 

- CMS Data Entries requirements. 

- CMS User Manual. 

(4) Users Training: User training is available as needed. OCR 
also provides training through a “peer helper” or “train the 
trainer” model for staff in the field offices and headquarters. 
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GOAL 5: CONTINUOUS IMPROVEMENT OF THE U.S. EDUCATION SYSTEM. Enhance the 
education system’s ability to continuously improve through better and more widespread 
use of data, research and evaluation, evidence, transparency, innovation, and technology. 


slag Fenermance Data Source(s) Data Verification and Validation Overview 

oO. Metric (Indicator) 

5.1.A Number of public Data Strategy The data are validated with a crosswalk between Inventory 
data sets included Team Data entries and the listing of the Department’s public datasets, 
in the Department's | Inventory and the ensuring that the data described in the Department's Data 
Data Inventory and | public Inventory are publicly available at the identified web address. 
thus linked to Department's Data 
Data.gov or Inventory at 
ED.gov websites http://datainventory 

.ed.gov 

5.1.B Number of states Statewide Data are collected through monitoring of states with active 
linking K-12 and Longitudinal Data SLDS grants. The data are limited to this population. While 47 
postsecondary Systems (SLDS) states and territories have received these grants over time, by 
data with workforce | grant monitoring June 2015, there will be fewer than 25 states with active 
data (monthly updates grants, which leads to incomplete and not up-to-date data from 

from states, annual | states that either have not received grants or that do not have 

performance active grants. 

reports, final If the state is focusing on an early childhood data system, the 

performance state’s reporting might reflect only those development efforts, 

reports, and site and not statewide P20W (preschool to college workforce) 

visits) development efforts. There are additional sources for 
information about state data linkages in the field. For example, 
the Data Quality Campaign (DQC) conducts surveys of state 
capacity to collect, store, link, and use data. Because the 
Department does not use the same definitions as DQC, our 
figures tend to be lower than theirs. 
For example, we require that a state possess the capacity to 
follow its own students in order for us to report that the state 
has the linkage in place. Similarly, we enable states to report 
on whether particular linkages are planned, in progress, or 
complete, and report that a state has a linkage when the state 
reports that the project is complete; DQC might give a state 
credit for an ‘in progress’ or pilot-stage linkage. 
A survey administered to the universe of states and territories 
would enable more a systematic collection data about all 
states’ capacity for data linkages and data use. There is a 
concern, however, that if those data were to be used for public 
reporting, states might begin to overstate their capacities, 
particularly on data that are also publicly reported by 
organizations such as DQC. 
Currently, data from monitoring are used in an iterative, 
formative approach to program improvement; our technical 
assistance program is designed to support states’ efforts to 
improve their systems. This relies on states being honest about 
their own internal capacities. 

5.1.C Number of states Statewide Data for this metric were independently assessed by SLDS 
actively using data | Longitudinal Data program staff using state progress reports, monitoring call 
systems to support | Systems (SLDS) summaries and site visit write-ups to determine which states 
and inform grant monitoring are actively using their data systems to inform improvements. 
improvements (monthly updates An operational definition for "active use” was agreed upon and 

from states, annual | is used consistently by SLDS staff to avoid bias. 

performance Definitions for active use are based upon tangible deliverables 
reports, final within the grant program. By explicitly counting the number of 
performance grantee states with deliverables that meet the definition’s 
reports, and site criteria we are able to accurately measure the explicit behavior 
visits) described in the metric. 
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Metric 
No. 


Performance 
Metric (Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


5.1.D 


Web traffic to the 
College Scorecard 


Google Analytics 
data from College 
Scorecard 


The data are simply pulled from the GSA Google Analytics 
account. The data are checked for reasonability, and 
incorporated alongside the existing baseline data. The data 
need to be matched with baseline data and collated 
accordingly. There is limited analysis required. 


5.2.A 


5.3.A 


Average time to 
close “cases” 
(PTAC + FPCO) 


Percentage of 
select new (non- 
continuation) 
discretionary grant 
dollars that reward 
evidence 


Case Tracking 
System (CTS) 
Monthly Metric 
Reports 


Forecast Report 
issued by the 
Office of the Chief 
Financial Officer 
(OCFO) and final 
Funding Reports 
from relevant 
programs 


The term “case” refers to requests for quick, informal 
responses to routine questions related to student privacy. 
These requests are received via email, the Family Policy 
Compliance Office (FPCO) / Privacy Technical Assistance 
Center (PTAC) resource website, or by telephone and 
subsequently entered into the CTS. In contrast, 
“correspondence and complaints” refers to written complaints 
of alleged failures to comply with Family Educational Rights 
and Privacy Act / Protection of Pupil Rights Amendment filed 
with FPCO; requests for formal written guidance/interpretation 
laws administered by FPCO; and, to the reporting of instances 
of data breaches by educational agencies and institutions. 
These inquiries are logged into the Correspondence Control 
Manager System, given a tracking control number, and 
assigned to FPCO staff. 

The preliminary data for this metric are reviewed at least 
weekly for verification. If anomalous data are identified in the 
periodic reviews or when anomalies are suspected, individual 
cases are examined individually to identify if they were properly 
closed or if their status was entered incorrectly. When 
appropriate, corrections are made. Staff responsible for 
entering data into the CTS will continue to be trained on 
policies and procedures. 

The monthly metric reports are scrutinized by the Director of 
FPCO, the Contracting Officer Representative for the PTAC 
contractors, and the Department's Chief Privacy Officer, to 
assure completeness and reliability of data and to recommend 
any improvements to the CTS or modifications to the standard 
operating procedures. The quarter entry represents the fiscal 
year to date average days to close as of the end of that quarter 
taken from the corresponding monthly report. 

In determining which discretionary grants are considered 
“evidence-based” (i.e., the numerator when calculating the 
percentage), the Department includes all programs that 
rewarded applicants with supporting evidence of promise or 
better (per the Education Department General Administrative 
Regulations (EDGAR) evidence framework). This could be 
done through a competitive preference or absolute priority, an 
eligibility requirement, or a selection factor. Only the amounts 
of the grants awarded for those projects were counted. In 
determining what counts as discretionary funding (i.e., the 
denominator when calculating the percentage), the Department 
includes all programs for which the EDGAR evidence 
framework could conceivably work. In Fiscal Year 2015, the 
Department counted all discretionary grant programs except for 
those programs run through the Institute of Education Sciences 
(IES), which are already evidence-based and would not be 


candidates for the EDGAR evidence framework). 
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slag Pen Hornance Data Source(s) Data Verification and Validation Overview 

oO. Metric (Indicator) 

5.3.D Number of Discretionary grant | We expect that it will take about five minutes to search the 
completed project slate memoranda, WWC database for new Investing in Innovation Program (i3) 
evaluations from discretionary grant | studies each quarter. We are also considering ways in which 
grantees of select financial forecasts IES staff and Oll staff can work closely to report these data 
discretionary grant | and reports from quickly. 
programs ina OCFO, and the We do not have formal verification or validation processes in 
given fiscal year What Works place. We do not have formal processes in place to check for 
that meet What Clearinghouse anomalous data. So far, we have not encountered anomalous 
Works data. 

Clearinghouse We note that we will report out on studies that are funded by i3 

(WWC) Evidence that are in the WWC database and meet WWC Evidence 

Standards Standards. We need to be careful not to imply that these 
studies all reflect positive results — in actuality, the WWC 
Evidence Standards only measure the rigor of impact 
evaluations and do not consider the actual outcomes reported. 

5.4.A Percentage of Education The Department uses an external data source for this metric 
schools in the Superhighway and relies on the external, third party’s verification and 
country that have validation methodology. We believe these data are the best 
actual internet currently available and provide useful information to gauge 
bandwidth speeds progress on this metric. ESH analyzes publicly available data 
of at least 100 from the Universal Service Administrative Company, an 
Mbps independent, not-for-profit corporation created by the Federal 

Communications Commission, confirms data accuracy through 
outreach and verification processes, and further analyzes the 
total IA bandwidth as compared to the National Center for 
Education Statistics data on student population in districts. 
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GOAL 6: U.S. DEPARTMENT OF EDUCATION CAPACITY. Improve the organizational 
Capacities of the Department to implement this strategic plan. 


Viewpoint Survey 
(FEVS) 


Metne Perro nTaniee Data Source(s) Data Verification and Validation Overview 
No. Metric (Indicator) 

6.1.A Percent of EDHires (Monster’s | The data are entered into the EDHires system by customers, 
selections made electronic hiring contractors and staff, so quality of data is only as reliable as 
per job opportunity | management the information entered and/or updated in the system. 
announcement system) Data are reviewed by HR Specialists for relevance and 

completeness to ensure correct parameters and filters have 
been applied. 

If anomalies are identified, HR Specialists will compare 
contractor generated reports against ad-hoc hiring reports 
generated from the system of record and other linked HR 
systems. Any questionable results would be brought to the 
contractor’ attention; in-turn, Office of Human Resources 
(OHR) would work with the contractor to obtain clarity and/or 
resolve. 

6.1.B EVS Employee OPM Federal Any questionable FEVS results would be brought to the 
Engagement Index | Employee attention of OPM; the Department would then work with the 


OPM point of contact to obtain clarity. 

The Engagement index score is calculated by OPM by first 
determining the percent positive for each of the 15 items in the 
sub-indices (i.e., Leaders Lead, Supervisors, and Intrinsic Work 
Experiences). Then the unrounded percent positive scores are 
averaged to obtain individual sub-index scores. The unrounded 
sub-index scores are then averaged to obtain the overall index 
score. The overall index score is then rounded for reporting 
purposes. 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 89 


APPENDICES 


Metric 
No. 


Performance 
Metric (Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


6.1.C 


Time to hire 


Federal 
Personnel/Payroll 
System (FPPS) 
Datamart 


The Office of Management (OM) identified a gap between 
hiring procedures and the 90 Day Hiring Model's time to hire 
calculations which necessitated an adjustment in the way time 
to hire is calculated for some actions. OM found that 
recruitment work is often initiated prior to the receipt of an 
“official” action. While this is a proactive customer-centric 
approach, especially prevalent in ED’s non-competitive hiring 
activities, it can sometimes result in a negative time to hire ora 
time to hire of zero — neither of which reflects a realistic or 
meaningful hiring lead time. 

To account for, track and document this upfront work, OM 
developed an adjustment mechanism to better gauge hiring 
lead time in these cases. For ease of comparison, the 
adjustment was applied to all FY16 hiring actions and resulted 
in a revised Q1 actual. 

To ensure clarity and consistency in the application of time to 
hire methodology moving forward, OM also clarified time to hire 
calculation protocols as follows: 

Individual time to hire: 

e Time to hire for individual hiring actions is determined 
by calculating the number of days between the 
Request to Recruit (ROE) Date and the Verbal 
(tentative) Offer Date, as reflected in FPPS DataMart. 

o If time to hire results in a negative or zero, it 
is replaced with the value 1. 

o If time to hire cannot be determined (i.e., 
date not available), it is excluded from 
agency calculations. 

e Time to hire is not calculated for actions In-Progress; 
it is only calculated for Completed actions (i.e., 
estimated EOD established). 

Agency time to hire: 

e The Department’s time to hire is an annual rate 
determined by calculating the percent of hiring actions 
within a fiscal year completed within 90 days. 

e §$Q1, Q2, and Q3 actuals are cumulative, measured 
from the beginning of the fiscal year. 

e Q4 actuals represent the Department's annual time to 
hire rate and are reported in the Annual Performance 
Report. 

e Periodic snapshots of time to hire may be computed 
monthly, quarterly or by principal office to help gauge 
progress toward the annual target; however, care 
should be taken to ensure periodic updates are clearly 
identified as snapshots or progress indicators and not 
represented as the agency’s annual rate. 


6.1.D 


Effective 
Communication 
Index 


Office of Personnel 
Management 
(OPM) Federal 
Employee 
Viewpoint Survey 
(FEVS) 


Any questionable FEVS results would be brought to the 
attention of OPM; the Department would then work with the 
OPM point of contact to obtain clarity. 

This index score is calculated by the Department by averaging 
the percent positive scores from OPM FEVS questions 53, 58, 
and 64. 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 90 


APPENDICES 


Metric 
No. 


Performance 
Metric (Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


6.2.A 


Percentage of 
Department grant 
recipients without 
any Single Audit 
Findings 


Federal Audit 
Clearinghouse 
(FAC) 


The FAC is the primary data source. OCFO and FSA import 
and manage Single Audits in Accountability and Audit 
Resolution Tracking System (AARTS) and EZ-Audit 
respectively. Reporting for this metric is based on downloading 
FAC data from their web portal. 

Simple calculations are made with data that are downloaded 
from FAC that include subtracting the total number of audits 
with findings from all audits received, then dividing to determine 
the overall percentage. We do not expect the calculations to be 
time consuming, so data can be reported to the Performance 
Improvement Office each quarter. However, quarterly data will 
be more variable than year-end data given the cycle of when 
audits are submitted to the FAC. 

Metric relies on public data from FAC database. The FAC 
maintains controls to verify and validate data, including 
procedures to review the completeness of audits before they 
are formally accepted. OCFO will obtain a copy of those 
procedures to support this metric. 

OCFO maintains an interagency agreement with the FAC and 
internal procedures to review and verify the accuracy of all 
audits with findings that are imported into AARTS for 
resolution. FAC staff manually review and correct coding errors 
in the audit data collection form. OCFO staff review and correct 
any errors in the imported audit file before it is assigned for 
resolution. These procedures are documented. Moving 
forward, OCFO also will use AARTS and EZ-audit data to 
reconcile the total number of audits that have findings with the 
total generated through the FAC report. 


6.2.B 


Compliance rate of 
contractor 
evaluation 
performance 
reports 


Past Performance 
Information 
Retrieval System 
(PPIRS) 
Compliance Metric 
Report 


Compliance rates of contractor performance evaluations are 
set by OMB and are calculated by use of a Government wide 
reporting tool available in the PPIRS (www.ppirs.gov). 

Office of the Chief Financial Officer (OCFO) staff analyzes the 
PPIRS report weekly to ensure that each contractor 
performance evaluation reflected on that report should appear 
on that report, and to rectify any errors on the report. 


6.3.A 


Overall average 
impact score of the 
Department’s 
technical 
assistance in 
helping build state 
capacity to 
implement 
education reforms 


Annual Grantee 
Satisfaction Survey 


The contractor has a quality control system and the results are 
also provided to all of the programs that participate in the 
survey to identify issues/anomalies with the data. 

Program staff report anomalies to the contractor to correct. 
The data comes from the Annual Grantee Satisfaction Survey 
Report (See 
http://www2.ed.gov/about/reports/annual/gss/index.html). The 
report is produced by a contractor; the contract is managed by 
OCFO. 

One major issue with the Grantee Satisfaction Survey is the 
timing of the survey. The survey is administered in late 
spring/early summer when many staff are out of the office 
which suppresses the response rate. Also, some grant 
programs do a better job than others of promoting the survey 
and showing how the survey results have been used to make 
improvements. We've been told that the survey is administered 
late each year for the following reasons: 

The Interagency Agreement can’t be put into clearance until 
the budget is approved each year, getting the Interagency 
Agreement through Departmental clearances takes several 
months, program staff are asked to review their customized 
survey questions and submit their grantee contact lists at the 
busiest time of the year which makes it difficult to turn them 
around quickly. 
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Metric 
No. 


Performance 
Metric (Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


6.4.A 


Number of the 
Department’s IT 
security incidents 


Cyber Security 
Assessment and 
Management 
(CSAM) and RSA 
Security 
Operations 
management 
(SecOps) 


Office of the Chief Information Officer (OCIO) has standard 
operating procedures (SOP) to verify and validate the data: 1) 
CSAM SOP, 2) Incident Response SOP, and 3) SecOps SOP. 
This process is executed on a weekly basis. Further, the data 
are pulled from the Department’s authoritative source (system 
of record) for Incident Response and Tracking. In addition, 
standard operate procedures (SOPs) enforce the use of this 
capability for entering, tracking, and managing all incident 
related activity. The data are also audited on a routine basis to 
ensure consistency and accuracy. 


6.4.B 


EVS Results 
Oriented 

Performance 
Culture Index 


OPM Federal 
Employee 
Viewpoint Survey 
(FEVS) 


Data verification and validation process is integrated into the 
OPM FEVS survey results validation process. The Results- 
Oriented Performance Culture Index score is calculated by 
OPM by first determining the percent positive for each of the 13 
items in the index. Then the unrounded percent positive scores 
are averaged across all index items to get the index score. The 
index score is then rounded for reporting purposes. Any 
questionable FEVS results would be brought to the attention of 
OPM; the Department would then work with the OPM point of 
contact to obtain clarity. 


6.4.C 


EVS Leadership 
and Knowledge 
Management Index 


OPM Federal 
Employee 
Viewpoint Survey 
(FEVS) 


Data verification and validation process is integrated into the 
OPM FEVS survey results validation process. The Leadership 
and Knowledge Management Index score is calculated by 
OPM by first determining the percent positive for each of the 12 
items in the index. Then the unrounded percent positive scores 
are averaged across all index items to get the index score. The 
index score is then rounded for reporting purposes. Any 
questionable FEVS results would be brought to the attention of 
OPM; the Department would then work with the OPM point of 
contact to obtain clarity. 


6.4.D 


Total usable 
square footage 


Department's 
Master Space 
Management Plan 


The Department reconciles its rent bills per building on a 
monthly basis. Every six months the usable square footage is 
verified with GSA. 

Because usable square footage relates directly to rent costs, 
the Department uses the same data verification and validation 
procedures. Specifically, the data are collected directly from 
Occupancy Agreements and rent bills per building. 

The data are derived from historic examples and relevant 
experience. Department leadership has agreed to a set of 
assumptions by which the data are based. Leadership has 
reached out to subject matter experts to broaden the scope of 
the data set, and lower risks of missing contingencies that may 
affect the data. At each step, the data are reviewed 
independently to double check the work of each team member 
and provide quality control. These processes help ensure the 
data’s completeness and reliability. 


6.4.E 


Rent cost 


Department's 
Master Space 
Management Plan 


Data are collected directly from Occupancy Agreements and 
rent bills per building. The actual rent may vary significantly if 
the Department relocates to a new leased building and/or signs 
short lease extensions. The Department is leveraging the 
examples and experience of the mobility labs and building 
consolidations programs. 

The Department reconciles its rent bills per building on a 
monthly basis. Every six months, leadership will re-evaluate the 
data, the assumptions on which it is based, and incorporate 
actual costs and project schedules. These steps will become 
part of our quality assurance program and procedures. 
Leadership looks to improve completeness, reliability, and 
quality of the data at these milestones. 
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Appendix C: Data Validity and Verification for FY 2019 
Performance Metrics 


The Government Performance and Results Act Modernization Act of 2010 requires agencies to 
describe the accuracy and reliability of data presented. Details of how the Department plans to 
assess the completeness and reliability of the data reported for the FY 2019 performance 
metrics are presented as part of this appendix, and known limitations of the data are also 
included. 


To the best of my knowledge, the data verification and validation process and the data sources 
used provide, to the extent practicable, complete, and reliable performance data pertaining to 
goals and objectives in our FY 2018-22 Strategic Plan. Through a process of continuous 
improvement, the Department regularly assesses its validation process and welcomes input 
from stakeholders. 


Joseph C. Conaty 
Senior Policy Advisor 


Office of the Deputy Secretary 
February 12, 2018 
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students in every community. 


AS ceed Performance Metric 
Strategic ; Data Source(s) Data Verification and Validation Overview 
ao (Indicator) 
Objective 
1.1.A Number of open and National Center Program officers will review and verify data as part 
operating charter schools for Education of their project monitoring. Program officers will 
supported by CSP Statistics, CCD review reported data and confirm the accuracy with 
1.1.B Number of students (used to identify the grantees. The Charter School Program (CSP) 
enrolled in charter schools all charter team will work closely with their contractor to 
supported by CSP schools and compare data in the CCD with data in grantees’ 
enrollees) and Annual Performance Reports (APRs) to confirm the 
grantee annual data. The CSP director is responsible for certifying 
performance that the data are accurate. 
reports (used to 
identify the 
subset of charter 
schools in CCD 
that receive CSP 
support) 
1.1.C Number of new resources National Charter | Verification and validation of data is based on 
on evidence-based and School Resource | review and approval of resources (promising 
promising practices related | Center and IES practices documents, papers, etc.). The CSP 
to school choice sponsored Director is responsible for certifying that the data 
disseminated materials are accurate. 
1.1.D Number of students Grantee annual The Department relies on a contractor to conduct 
enrolled in magnet schools | performance data reviews of grantee performance data. The 
reports contractor is responsible for locating, assessing, 


and recording annual performance data. Review 
protocols include checking and documenting the 
presence and completeness of performance 
measure data for analysis. The contractor 
maintains databases with classification and coding 
schemes in Microsoft Excel and SPSS to analyze 
grantee data, including student enrollment. The 
contractor addresses anomalous data during the 
data verification and validation process. The 
contractor’s research staff discusses anomalous 
data with grantees, who verify or correct the data. If 
the data are correct as reported, the contractor will 
document and clarify the anomaly in the report. The 
Parental Options and Improvement Programs (POI) 
Director is responsible for certifying that the data 
are accurate. 
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show improvement in the 
percentage of students in 
grades 3-8 scoring at or 
above proficient on state 
assessments in reading in 
all of the following 
subgroups: economically 
disadvantaged, children 
with disabilities, English 
learners, migrant, 
homeless, and major 
racial/ethnic groups 


File that includes 
state reported 
data pulled from 
EDFacts files 
C175, C178, 
C185, and C188 


Assorted Performance Metric 
Strategic A Data Source(s) Data Verification and Validation Overview 
nae (Indicator) 
Objective 
1.2.A Percentage of states that Assessment Data | The Office of Elementary and Secondary Education 


(OESE), Office of Special Education Program 
(OSEP), and EDFacts conduct a thorough, 
coordinated data quality review of the assessment 
data submitted by states. OSEP reviews this set of 
assessment data files for the purposes of IDEA 
Section 618 and OESE reviews this set of 
assessment data files for the purposes of the 
Consolidated State Performance Report (CSPR). 
The review includes the following three areas: 
timeliness of the data submission, completeness of 
the data files, and accuracy of the data. 

OESE, OSEP, and EDFacts conduct three data 
quality reviews of states’ assessment data 
submissions. The first two data quality reviews 
result in data quality inquiries, comments, and 
questions that are sent to the states for responses. 
A state may resubmit the data to address the data 
quality inquiries, provide a written response, or a 
data note to address the data quality inquiry. The 
three-data quality review is conducted to ensure 
that the Department is publishing and using a set of 
assessment data that meets our data quality 
criteria. 

Additionally, OESE, OSEP, and EDFacts will hold 
conference calls with states to gain better 
understanding of anomalies that are identified in 
the data submissions or to provide technical 
assistance to the state to submit higher quality 
assessment data 

These data quality reviews are conducted after 
each of the due date/ resubmission dates for these 
data. If there are questions regarding whether the 
data submitted by a state are accurate remain then 
the Department may decide not to publish or use 
the data. 

Note that this metric is impacted by changes to 
state assessment systems. If states change 
assessments, performance levels, cut scores, etc., 
it will invalidate the year to year analysis to identify 
states that showed improvements in the 
percentage of students proficient. Since statewide 
assessment systems have been influx for the last 
few years, it makes it challenging to establish 
baselines and set targets. 

States sign a certification when submitting their 
CSPR. 

If questions remain upon completion of the 
Department’s data quality review process regarding 
whether the data submitted by a state are accurate, 
the Department may decide not to publish or use 
the data. 

Note that this metric is impacted by changes to 
state assessment systems. If states change 
assessments, performance levels, cut scores, etc., 
it will invalidate the year to year analysis to identify 
states that showed improvements in the 
percentage of proficient students. Since statewide 
assessment systems have been influx for the last 
few years, it makes it challenging to establish 
baselines and set targets. 
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show improvement in the 
percentage of students in 
grades 3-8 scoring at or 
above proficient on state 
assessments in 
mathematics in all of the 
following subgroups: 
economically 
disadvantaged, children 
with disabilities, English 
learners, migrant, 
homeless, and major 
racial/ethnic groups 


File that includes 
state reported 
data pulled from 
EDFacts files 
C175, C178, 
C185, and C188 


Associated Performance Metric 
Strategic A Data Source(s) Data Verification and Validation Overview 
os (Indicator) 
Objective 
1.2.B Percentage of states that Assessment Data | OESE, OSERS, and EDFacts conduct a very 


thorough CDQR of the assessment data submitted 
by states. OSERS reviews this set of assessment 
data files for the purposes of IDEA Section 618 and 
OESE reviews this set of assessment data files for 
the purposes of the CSPR. The review includes the 
following three areas: timeliness of the data 
submission, completeness of the data files, and 
accuracy of the data. Through the coordinated 
review, the states receive one set of data quality 
comments or inquiries associated with the 
assessment data from the Department. 

OESE, OSERS, and EDFacts conduct three data 
quality reviews of states’ assessment data 
submissions. The first two data quality reviews 
result in data quality inquiries, comments, and 
questions that we sent back to states for 
responses. A state may resubmit the data to 
address the data quality inquiries and/or provide a 
written response and/or a data note to address the 
data quality inquiry. The three-data quality review is 
conducted to ensure that we are publishing and 
using a set of assessment data that meets our data 
quality criteria. 

These data quality reviews are conducted after 
each of the due date/ resubmission dates for these 
data. Typically, these due date/ resubmission dates 
occur in December, February/ March, and April. 
OESE, OSERS, and EDFacts identify anomalous 
data via the coordinated data quality review noted 
in question 11 (above). Once anomalous data is 
identified via the data quality review and analyses, 
states are given the opportunity to resubmit the 
data or provide an explanation for the anomalous 
data. Additionally, OESE, OSERS, and EDFacts 
will hold conference calls with states to gain better 
understanding of anomalies that are identified in 
the data submissions and/or to provide technical 
assistance to the state to submit higher quality 
assessment data. 

If there are questions regarding whether the data 
submitted by a state are accurate after the mid- 
April freeze date, OESE and/or OSERS may 
decide not to publish or use the data. 

This measure will be impacted by changes to state 
assessment systems. If states change 
assessments, performance levels, cut scores, etc., 
it will invalidate the year to year analysis to identify 
states that showed improvements in the 
percentage of students proficient. Since statewide 
assessment systems have been influx for the last 
few years, it makes it challenging to establish 
baselines and set targets. 

For states that have missing or inaccurate data, 
there are opportunities to resubmit their data files 
and have them reviewed prior to publication. 
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Assorted Performance Metric 
Strategic 2 Data Source(s) Data Verification and Validation Overview 
Obiective (Indicator) 

1.2.C Percentage of states with IDEA Part B OSEP State Leads review each states’ data for 
80% or more of State Annual data quality issues once the annual performance 
preschoolers with Performance report is received. The MSIP Data Implementation 
disabilities that showed Reports Team look across all states to review the data for 
greater than expected any anomalies. OSEP’s technical assistance 
growth in positive social- center, which has expertise on early childhood 
emotional skills (including outcome measurement systems, also reviews the 
social relationships) by the data for quality issues. The center publishes a brief 
time they exit IDEA Part B, on how they analyze the data 
section 619 services http://ectacenter.org/eco/. The number of states 

collecting high-quality data has increased over 
time. In addition, states continue to build their 
capacity to collect valid and reliable data. These 
efforts are supported by the aforementioned TA 
center, which helps states build and improve their 
outcome measurement systems, collect and 
analyze data, and use data to make program 
improvements. States certify that the data they turn 
in to OSEP is accurate. The OSEP Director signs 
the determination letter for each state. 

1.2.D Percentage of schools in Education Verification and validation of data is managed by 
the country that have actual | Superhighway Education Superhighway. 
internet bandwidth speeds https://s3-us-west-1.amazonaws.com/esh-sots- 
of at least 100 Mbps pdfs/methodology 2017 state of the states.pdf. 

1.2.E Percentage of rural schools The process for checking for anomalous data is 


connected to a broadband 
infrastructure capable of 
scaling to 10 gigabits per 
second 


Education Superhighway. The Office of Education 
Technology (OET) is responsible for certifying that 
the data are accurate. 
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Associated 
Strategic 
Objective 


Performance Metric 
(Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


1.2.F 


Percentage of states 
publishing report cards on 
the preceding school year 
in a timely manner* 


*Timeliness defined as 
report cards that are 
published by January 15th 
of the year following the 
reporting year (e.g., report 
cards on the 2017-18 
school year would be timely 
if published by January 15, 
2019) 


Consolidated 
State 
Performance 
Report (CSPR) 


The data will be collected through the CSPR. There 
is a formal, extensive process for the verification 
and validation of any information submitted through 
EDFacts/CSPR which occurs annually, beginning 
immediately following the submission due date, ina 
series of review windows between December and 
May. Data relevant to OSS are reviewed by OSS 
program staff, and the results of the process are 
further reviewed by the OSS Data Team, OESE 
Front Office, and EDFacts team. All staff engaged 
in this process are trained in the content and review 
process for these data. Due to the scope and 
complexity of the process, most data included in 
the CSPR are not considered final and available for 
use until May. However, since the review of the 
report card link will only involve navigating to the 
web location and confirming that a current report 
card is posted, OSS will be able to use preliminary 
data, which will be available earlier in the year, in 
order to produce this metric. Data will be reviewed 
by OESE prior to being submitted. Anomalous data 
will be checked and verified by OESE or other 
supporting offices. The ESSA report card data 
metrics are new to states and some states may be 
working through the programming necessary to 
report them and therefore may be delayed for the 
first couple of years of implantation in the release of 
their data. However, the data to be collected by the 
Department via CSPR should be standard. States 
are responsible for certifying the accuracy of data 
submitted through the CSPR. The data are further 
reviewed by OSS to identify any anomalies before 
being used in any public products. 


1.2.G 


Percentage of monitored 
states reporting Information 
on each indicator in the 
state’s accountability 
system and the list of 
schools identified for 
comprehensive support and 
improvement and targeted 
support and improvement. 


OESE Title | 
monitoring 
reports of ESSA 
State report cards 


The data will be collected during Title | monitoring 
by OESE. Monitoring reports are reviewed both 
internally by staff in OSS and OESE and submitted 
to states as part of the monitoring process. 
Program officers doing the monitoring are trained 
for consistent and standard application of 
monitoring protocols. Data will be reviewed by 
OESE prior to being submitted. Anomalous data 
will be checked and verified by OESE or other 
supporting offices. The ESSA report card data 
metrics are new to states and some states may be 
working through the programming necessary to 
report them. However, the data to be collected by 
ED via monitoring should be standard. OESE is 
responsible for monitoring the Title | program and 
follow the protocols necessary to ensure the data 
are accurate. 
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Assorted Performance Metric 
Strategic 2 Data Source(s) Data Verification and Validation Overview 
Obiective (Indicator) 

J 

1.3.A Number of discretionary Program offices Reports can be checked against the NIA for each 
grant notices with STEM as _ | holding competition. Each program office listed in the data 
a priority discretionary source is responsible for certifying that the data are 

grant accurate. 
competitions 

each year 

(including OESE, 

Oll, OSERS, 

OPE, OCTAE, 

IES, and OELA) 

1.3.B Number of public high College Board College Board addresses all data quality issues 
school graduates who have and is responsible for certifying that all data are 
taken at least one accurate. 

Advanced Placement (AP) The College Board does not collect or report 
Science, Technology, race/ethnicity based on the federal guidelines. 
Engineering, Mathematics Examinees are asked to select one of the options 
(STEM) exam’ while in high noted in the data. The College Board Public School 
school List is updated annually by state educational 

agencies; thus, small changes to the list over time 
*Including the following are to be expected as schools open, close, and/or 
exams: Biology, Chemistry, merge. Students are assigned to graduating 
Environmental Science, cohorts based on self-reported information (i.e., 
Physics B, Physics C: grade level and/or graduation year) provided at the 
Electricity and Magnetism, time of registration (in the case of SAT) or test 
Physics C: Mechanics administration (in the case of AP and PSAT). The 
Based, Calculus AB, College Board matches students’ data across 
Calculus BC, Computer programs to identify the most recent valid value 
Science A, and Statistics when assigning students to cohorts. 

1.3.C Number of public high College Board College Board addresses all data quality issues 
school graduates who have and is responsible for certifying that all data are 
taken at least one accurate. 

Advanced Placement (AP) The College Board does not collect or report 
Science, Technology, race/ethnicity based on the federal guidelines. 
Engineering, Mathematics Examinees are asked to select one of the options 
(STEM) exam’ while in high noted in the data. The College Board Public School 
school and score a 3 or List is updated annually by state educational 
better agencies; thus, small changes to the list over time 
are to be expected as schools open, close, and/or 
*Including the following merge. Students are assigned to graduating 
exams: Biology, Chemistry, cohorts based on self-reported information (i.e., 
Environmental Science, grade level and/or graduation year) provided at the 
Physics B, Physics C: time of registration (in the case of SAT) or test 
Electricity and Magnetism, administration (in the case of AP and PSAT). The 
Physics C: Mechanics College Board matches students’ data across 
Based, Calculus AB, programs to identify the most recent valid value 
Calculus BC, Computer when assigning students to cohorts. 
Science A, and Statistics 

1.3.D Percentage of adult NRS for Adult Office of Career, Technical and Adult Education 
education participants in Education (OCTAE) provides guidance and technical 
adult basic education assistance to states on data validity, reliability, and 
programs who achieve a completeness. States are required to certify data 
measurable skill gain quality annually. Sufficient edits checks are a 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 


99 


APPENDICES 


Resorts Performance Metric 
Strategic 2 Data Source(s) Data Verification and Validation Overview 
Obiective (Indicator) 

J 

1.3.E Percentage of adult requirement that must be addressed as part of the 
education participants who process for states certifying data quality. States 
obtain a secondary school with an overall data quality level that falls below 
diploma or its equivalent “superior” must submit a data quality improvement 
and are employed or plan. OCTAE conducts ongoing desk and on-site 
enrolled in an education or monitoring to verify data quality. The director of the 
training program within one Division of Adult Education and Literacy in OCTAE 
year following exit is responsible for ensuring that the data are 

accurate. 

1.3.F Percentage of secondary State OCTAE performs a 10-step check for validity, 
career and technical Consolidated reliability, and completeness of data. Under the 
education (CTE) Annual Reports current Carl D. Perkins Career and Technical 
concentrators who attain a (CARs) for the Education Act, states have the authority to 
secondary school diploma, Carl D. Perkins determine definitions of CTE concentrator and the 
a General Education Career and definitions of their measures. Director of the 
Development credential or Technical Division of Academic and Technical Education 
other state-recognized Education Act (DATE) in OCTAE is responsible for certifying that 
equivalent (including the data are accurate. 
recognized alternative 
standards for individuals 
with disabilities), or a 
proficiency credential, 
certificate, or degree, in 
conjunction with a 
secondary school diploma 

14.A Number of technical Department of The Department has established criteria to 
assistance engagements, Education offices | accommodate the range of internal and external 
events or related activities that deliver technical assistance provided across offices. 
or products focused on the | technical Establishing a common definition of key terms and 
grantees’ use of evidence assistance program offices applying such definitions 
in kindergarten through consistently and in a standard way will be a 
grade 12 education challenge. PPSS will follow up with program offices 

as necessary to address any anomalous data. This 
metric provides descriptive information and no 
causal relationship could be attributed to the data 
and the performance of grantees. Contacts will be 
identified in individual program offices to be 
responsible for certifying accuracy of the data. 
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Associated : 
Strategic Peron aiee Mew Data Source(s) Data Verification and Validation Overview 
nar (Indicator) 
Objective 
2.1.A Percentage of OPE grantees | The The universe for this metric includes Higher 
with large available balances | Department's Education Program (HEP) grants and select 
Grants International and Foreign Language Education 
Management (IFLE). G5 data are historically accurate. Grantees 
2183 Percentage of OPE grantees System G5 tell program Staff when they think there is a 
with large available balances discrepancy in the balances. This is an independent 
that received technical source of validation. OPE managers will spot check 
assistance resulting in data and address anomalies with program staff. 
“Rasolved with Good Given that program cycles are in different stages 
Explanation” from year to year and each program is unique, the 
data for this metric may not be comparable each 
quarter or from year to year. The Deputy Assistant 
Secretary of HEP and the Assistant Secretary of 
OPE are responsible for certifying that the data are 
accurate. 
2.1.C Percentage of annual G5 will be updated to list statutory requirements for 
statutory requirements for each program. Managers will certify fulfillment. 
OPE programs that are Directors will spot check data and address 
fulfilled by OPE anomalies with managers. Office of the General 
Counsel (OGC) will be consulted if there is a 
question about whether a requirement has truly been 
fulfilled. The Deputy Assistant Secretary of HEP and 
the Assistant Secretary of OPE are responsible for 
certifying that the data are accurate. 
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Associated 
Strategic 
Objective 


Performance Metric 
(Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


2.1.D 


2.1.E 


Percentage of First-Time 
FAFSA Filers Among High 
School Seniors 


Persistence Among First- 
Time Filing Aid Recipients 


FSA’s Central 
Processing 
System 


FSA’s Common 
Origination and 
Distribution 
system 


performance from an earlier fiscal year. 


The data we use originate from FSA operational 
systems. These systems, as a part of their 
operations, have procedures in place to address 
potential data quality issues. These metrics report 
figures about data that our operational systems 
maintain, and we do not manipulate it in any way. If 
anomalies persist, they exist in the operational 
systems. 

We have a consistent and disciplined process in 
place for querying our systems for data. The process 
includes having a separate data analyst from a 
different office within FSA validate the accuracy of 
the query and the resulting data. This process also is 
used for anomalous data. 

In addition to the above controls, queries and 
calculations are simultaneously conducted on data 
from previous years to ensure technical definitions 
remain consistent and calculations by the business 
intelligence team as part of the validation. 

The Customer Analytics Group is responsible for the 
primary calculation of the metric using the data from 
FSA. Business Intelligence is responsible for the 
technical validation of the metric, which is done by 
reviewing for accuracy the query used to pull the 
data and all calculations made with the data. Finally, 
the Financial Reporting and Analysis Branch is 
responsible for ensuring that documentation is 
complete and archived. 

Since year of high school graduation is not asked on 
the FAFSA, several assumptions are made and 
specific criteria were used (Such as age of applicant) 
to identify those likely to be high school seniors. For 
the 2018-19 FAFSA, which was available beginning 
October 1, 2017, the age assumption for high school 
seniors was increased from 18 to 19 years of age. 
Using the age criteria of 18 years of age (used in 
previous years) resulted in FSA undercounting high 
school seniors by approximately 23%. Relaxing the 
age assumption to include FAFSAs from slightly 
older high school seniors produces more accurate 
data that also better reflect nationwide awareness 
and outreach efforts. 

These calculations also restrict the application period 
to the first nine months of the application cycle (the 
close of the fiscal year), rather than the entire 18 
months. Since most applicants, including high school 
seniors, file their FAFSA prior to the start of the 
upcoming academic year (usually before fiscal year 
end), this decision better aligns the performance 
metric with the fiscal year where most of the 
performance occurred. The alternative is waiting for 
the close of the 18-month cycle, where a 
performance metric would mostly reflect 


Data used for these calculations are based on 
counts from operational systems (number of 
recipients and number of applicants). Moreover, 
standardized queries are used to rerun and match 
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(one 3 Performance Metric 
Strategic é Data Source(s) Data Verification and Validation Overview 
ras: (Indicator) 
Objective 
calculations for earlier cycles. Therefore, the metric 
as defined should be considered verified and valid. 
2.2.A Number of technical Department of Criteria for and examples of technical assistance 
assistance events or Education events/activities and products are provided above. 
activities and products offices that The examples provided are not exhaustive, so 
focused on the use of deliver technical | program offices may be able to “count” additional 
evidence in federal assistance events/activities and products that meet the other 
programs that promote criteria. The criteria are flexible to accommodate 
educational opportunities, different technical assistance offerings provided 
training, and support across the Department. PPSS will follow up with 
services for the workforce. program offices as necessary to address any 
anomalous data. Establishing a common definition of 
key terms and program offices applying such 
definitions consistently and in a standard way will be 
a challenge. ED could conduct a few “level setting” 
meetings with staff charged with reporting from each 
program office to try to support collection of standard 
and meaningful data. In addition, this will be 
descriptive information and no causal relationship 
could be attributed to the data and the performance 
of grantees. Contacts will be identified in Individual 
program offices to be responsible for certifying 
accuracy of the data. 
2.2.B Percentage of adult NRS for Adult OCTAE provides guidance and technical assistance 
education program Education to states on data validity, reliability, and 
participants who are in completeness. States are required to certify data 
unsubsidized employment quality annually. Sufficient edits checks are a 
during the second quarter requirement that must be addressed as part of the 
after exit from the program process for states certifying data quality. States with 
an overall data quality level that falls below 
“superior” must submit a data quality improvement 
plan. OCTAE conducts ongoing desk and on-site 
monitoring to verify data quality. The director of the 
DAEL in OCTAE is responsible for ensuring that the 
data are accurate. 
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(eeonatd Performance Metric 
Strategic é Data Source(s) Data Verification and Validation Overview 
bao (Indicator) 
Objective 
2.2.C Percentage of VR RSA 911 The numerator for this metric is the participants who 
participants who are in Vocational were employed in the 2nd quarter after program exit 
unsubsidized employment Rehabilitation (RSA-911 Element 383). 
during the second quarter Case Service The denominator for this metric is the total number of 
after exit from the program Report participants who exited during the reporting period 


(RSA-911 Element 355). 

RSA will maintain a comprehensive edit check table 
on its website. The table will detail, by data element, 
the edit checks required to ensure the integrity of 
data submissions. Edits describe constraints that 
should be satisfied by the data. Each data 
submission will be analyzed to determine whether 
the data are consistent with the edit checks. Data 
submissions that fail to pass the edit check will be 
returned to the VR agency for correction and 
resubmission. Any corrections made must be 
consistent with the agency’s electronic case 
management system and the supporting 
documentation maintained by the agency. RSA’s 
data editing process will utilize both hard and soft 
edits. Hard edits identify records that “fail” based 
upon erroneous combinations or missing values. 
Soft edits are constraints that identify (combinations 
of) values that are suspicious but not necessarily 
incorrect. RSA data staff conduct macro-level checks 
to uncover inconsistencies in the data reported by 
state VR agencies (e.g., formatting errors, necessary 
relationships between data elements, and 
completeness of submissions). RSA will request 
agencies investigate and correct any anomalies in 
subsequent report submissions. Data for this metric 
are based on unemployment insurance data from the 
state’s Unemployment Insurance Tax System. There 
is a two-quarter lag in the availability of those data. 
As a result, the employment rate data reported is, in 
effect, reported four quarters after the individual exits 
the program. VR agencies are responsible for 
ensuring that any data submitted conforms to edit 
check and data submission requirements. VR 
Directors, or individuals formally delegated the 
authority to submit the data on behalf of the VR 
Directors, are required to certify the accuracy of the 
data that are submitted. 
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(eeonad Performance Metric 
Strategic é Data Source(s) Data Verification and Validation Overview 
raat: (Indicator) 
Objective 

2.2.D Percentage of VR program RSA 911 The numerator for this metric is the unique count of 
participants who, during a Vocational the most recent date on which participants achieved 
program year, are in an Rehabilitation a Measurable Skill Gains in the reporting period, via 
education or training Case Service one of the following: 
program that leads to a Report *Educational Functioning Level (RSA-911 
recognized postsecondary Element 343) 
credential or employment ePostsecondary Transcript/Report Card (RSA- 
and who are achieving 911 Element 345) 
measurable skill gains, eSecondary Transcript/Report Card (RSA-911 
defined as documented Element 344) 
academic, technical, *Training Milestone (RSA-911 Element 346) 
occupational, or other forms Skills Progression (RSA-911 Element 347) 
of progress, towards sucha The denominator for this metric is all participants 
credential or employment enrolled in an education or training program leading 

to a recognized postsecondary credential or 
employment (RSA-911 Element 85). 

RSA will maintain a comprehensive edit check table 
on its website. The table will detail, by data element, 
the edit checks required to ensure the integrity of 
data submissions. Edits describe constraints that 
should be satisfied by the data. Each data 
submission will be analyzed to determine whether 
the data is consistent with the edit checks. Data 
submissions that fail to pass the edit check will be 
returned to the VR agency for correction and 
resubmission. Any corrections made must be 
consistent with the agency’s electronic case 
management system and the supporting 
documentation maintained by the agency. RSA’s 
data editing process will utilize both hard and soft 
edits. Hard edits identify records that “fail” based 
upon erroneous combinations or missing values. 
Soft edits are constraints that identify (combinations 
of) values that are suspicious but not necessarily 
incorrect. RSA data staff conduct macro-level checks 
to uncover inconsistencies in the data reported by 
state VR agencies (e.g., formatting errors, necessary 
relationships between data elements, completeness 
of submissions). RSA will request agencies 
investigate and correct any anomalies in subsequent 
report submissions. VR agencies are responsible for 
ensuring that any data submitted conforms to edit 
check and data submission requirements. VR 
Directors, or individuals formally delegated the 
authority to submit the data on behalf of the VR 
Directors, are required to certify the accuracy of the 
data that are submitted. 

2.3.A Number of technical Quarterly The contractor progress reports include, as 
assistance activities, Progress applicable, links to virtual events and meeting 
sponsored by the Reports from materials from face-to-face events. OCTAE staff vet 
Department, intended to Contractor all event materials prior to the actual event. For 
expand or enhance the virtual events, contractor provides relevant analytics 
integration of workforce as supporting documentation. For face-to-face 
preparation activities within events, contractor collects sign-in sheets and event 
academic instruction in adult evaluation forms. OCTAE staff review contractor 
education classrooms documentation for anomalous or unclear 

submissions and follow up with the appropriate 
contractor. The Deputy Director of DAEL in OCTAE 
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Associated 
Strategic 
Objective 


Performance Metric 
(Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


is responsible for certifying that the data are 
accurate. 


2.3.B 


Percentage of adult 
education participants 
enrolled in an integrated 
education and training (IET) 
program 


NRS for Adult 
Education 


OCTAE provides guidance and technical assistance 
to states on data validity, reliability, and 
completeness. States are required to certify data 
quality annually. Sufficient edits checks are a 
requirement that must be addressed as part of the 
process for states certifying data quality. States with 
an overall data quality level that falls below 
“superior” must submit a data quality improvement 
plan. OCTAE conducts ongoing desk and on-site 
monitoring to verify data quality. The director of the 
DAEL in OCTAE is responsible for ensuring that the 
data are accurate. 


2.3.C 


Percentage of 
postsecondary CTE 
concentrators who received 
an industry-recognized 
credential, a certificate, or a 
degree 


State 
Consolidated 
Annual Reports 
(CARs) for the 
Carl D. Perkins 
Career and 
Technical 
Education Act 


OCTAE performs a 10-step check for validity, 
reliability, and completeness under the current Carl 
D. Perkins Career and Technical Education Act, 
states have the authority to determine the definitions 
of CTE concentrator and the definitions of the 
measures and so performance data may not be 
comparable across states. DATE in OCTAE, is 
responsible for certifying that the data are accurate. 


2.4.A 


Number of non-default 
federal student loan 
servicers’ call centers with 
expanded hours 


2.4.B 


Number of call centers that 
meet or exceed the quality 

standard for average speed 
to answer (ASA) 


Federal 
servicers 
quarterly reports 


Data will be collected from federal servicers through 
non-default federal student loan servicers’ quarterly 
reports. For this metric the query will be “how many 
call centers have expanded hours meeting or 
exceeding the defined times?” The verification and 
validation of performance by the non-default federal 
student loan servicers’ will be conducted by Federal 
Student Aid (FSA) including, but not limited to, (a) 
review and validation of federal servicer reports, (b) 
Ongoing/recurring quality assurance discussion with 
federal servicers, (c) site visits to federal servicer call 
center sites, and (c) documented on-phone 
(“mystery caller”) evaluations of services. Because 
the agency directive is succinct and builds upon 
current contractor operational capabilities, FSA does 
not anticipate anomalous data or issues with 
implementation. However, in cases where 
verification and validation detect anomalies that 
suggest less than complete information, FSA will 
address any deficiencies through direct contact with 
federal servicers, requests for information, audits, 
site visits, and/or other assessment measures of 
performance, as applicable. FSA is not presently 
aware of any limitations or constraints of the data. 
Business Operations Officer, FSA Business 
Operations, which oversees all of FSA’s non-default 
federal student loan servicers, is responsible for 
certifying that the data are accurate. 


Data will be collected from non-default federal 
student loan servicers through servicers’ quarterly 
reports. For this metric, the query will be “how many 
servicers met the standards for quality?” The 
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Associated 
Strategic 
Objective 


Performance Metric 
(Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


2.4.C 


Number of call centers that 
meet or exceed the quality 
standard for average 
abandon rate (AR) for 
incoming calls 


verification and validation of performance by the 
non-default federal student loan servicers will be 
conducted by FSA including, but not limited to, (a) 
review and validation of federal servicer reports, (b) 
ongoing/recurring quality assurance discussion with 
federal servicers, (c) site visits to federal servicer call 
center sites, and (c) documented on-phone 
(‘mystery caller’) evaluations of services. Because 
the agency directive is succinct and builds upon 
current contractor operational capabilities, FSA does 
not anticipate anomalous data or issues with 
implementation. However, in cases where 
verification and validation detect anomalies that 
suggest less than complete information, FSA will 
address any deficiencies through direct contact with 
federal servicers, requests for information, audits, 
site visits, and/or other assessment measures of 
performance, as applicable. FSA is not presently 
aware of any limitations or constraints of the data. 
Business Operations Officer, FSA Business 
Operations, which oversees all of FSA’s non-default 
federal student loan servicers, is responsible for 
certifying that the data are accurate. 


2.4.D 


ACSI Aid Life Cycle Surveys 


FSA’s Customer 
Satisfaction 
Surveys 


Traditionally, the ACSI Aid Life Cycle metric has 
been based on the following proportional weights: 

.65 — Borrower (in Repayment) Satisfaction — based 
on a survey of ED held Serviced Borrowers 
.29 — Applicant Satisfaction — based on the FAFSA 
on the Web Survey of Applicants who complete the 
FAFSA 
.06 — Borrower (in School) Satisfaction — based on 
our survey of borrowers still enrolled. 

For 2017, our Borrower in School Survey was not 
funded while applicants were still in school, so the 
survey could not be conducted. As a result, last 
year’s score was based on the following proportional 
weights: 

.76 — Borrower (in Repayment) Satisfaction — based 
on a survey of ED-held Serviced Borrowers 
.24 — Applicant Satisfaction — based on the FAFSA 
on the Web Survey. 

FSA plans to fund the 2018 Borrower in School 
survey this winter/spring in order to be able to 
conduct the Borrowers in School Survey and use 
that information as part of this metric for 2018. The 
ACSI Survey results have declined in the past few 
years, partly attributed to unexpected interruptions 
with the IRS Data Retrieval Tool during FAFSA 
completion. While the Borrower in School Survey is 
an important part of the overall Customer 
Satisfaction Survey, its inclusion is not a significant 
contributor to the overall score. Therefore, FSA 
expects the FY2018 ACSI Aid Life Cycle Survey 
score to be largely unchanged for the coming year. 
As FSA approaches changes to the FAFSA, we do 
expect sharp increases in customer satisfaction in 
future years. 
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that are more than 90 days 
delinquent 


Resonate’ Performance Metric 
Strategic : Data Source(s) Data Verification and Validation Overview 
beet: (Indicator) 
Objective 
2.5.A Percentage of borrowers FSA's Data Data will be collected on a quarterly basis from 
that are more than 30 days Warehouse FSA’s Data Warehouse. The verification and 
delinquent validation of the rate(s) of delinquency will be 
conducted by FSA’s Office of Performance 
25B Percentage of borrowers Management. Because the query has been 


performed before, successfully, FSA does not 
anticipate anomalous data or issues with 
interpretation. However, in cases where verification 
and validation detect anomalies that suggest less 
than complete information, FSA will address any 
deficiencies through collaboration with subject matter 
experts within the Office of Performance 
Management. No limitations, other than macro- 
economic situations (i.e., economic downturn), have 
been identified. Deputy Chief Operating Officer, FSA 
is responsible for certifying that the data are 
accurate. 
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better management, increased privacy protections and transparency. 


Agsoaniee Performance Metric 
Strategic A Data Source(s) Data Verification and Validation Overview 
wae (Indicator) 
Objective 
3.1.A Number of data Minutes from At the time minutes from the Monthly Meeting are 
management activities for Monthly DST submitted by the DST Support contractor to the DST 
which Department-wide meetings Co-Chairs, the information presented in support of 
procedures or templates this metric will be reviewed and accepted by the DST 
have been created and Co-Chairs. These meetings occur every two-weeks 
reviewed through the Data providing both the DST Support contractor and the 
Strategy Team (DST) DST Co-Chairs with an opportunity to review and 
3.1.B Number of the Department’s finalize documentation of the Meeting in a timely 
program offices participating fashion, and before a subsequent DST Monthly 
in DST offered data Meeting is held. Any data identified by the DST Co- 
management trainings Chairs as anomalous would be discussed at one of 
the bi-weekly meetings between the Co-Chairs and 
the DST Support contractor. Anomalous data would 
be researched using the original meeting notes from 
the monthly meeting in question and followed up with 
calls from the Co-Chairs to any points of contact 
whose approval or whose attendance at the Monthly 
Meeting was in question. The DST Co-Chairs are 
responsible for certifying that the data are accurate. 
3.2.A Number of Institutions of IHE-provided The data will be verified and validated by the FSA 
Higher Education (IHE) that auditor reports Senior Advisor for Cybersecurity in collaboration with 
have an audit of GLBA- the Department’ Chief Privacy Officer, on at least a 
related information security quarterly basis, and compared to report data, FSA 
safeguards which result in records, and ongoing compliance and investigations 
no significant findings regarding data breaches. Anomalous data resulting 
from GLBA-related audits will be brought to the 
attention of the auditing entities and reconciled 
where possible. There are no identified nuances or 
limitations to the data. FSA’s Deputy Chief Operating 
Officer will certify that all data are accurate. 
3.2.B Number of outreach Outreach activity | The data will be verified and validated by the FSA 
activities targeting data records Senior Advisor for Cybersecurity in collaboration with 
privacy and IT security maintained by the Department’s Chief Privacy Officer on at least a 
requirements of institutions the Privacy quarterly basis, and compared to report data, FSA 
of higher education Technical records, and ongoing compliance and investigations 
Assistance regarding data breaches. Anomalous data related to 
Center outreach activities will be noted and staff and 
contractors will be questioned to ensure that 
outreach activities and programs fall within the 
defined, qualifying activities. There are no identified 
nuances or limitations to the data. FSA’s Deputy 
Chief Operating Officer will certify that all data are 
accurate. 
3.2.C Percentage of LEA websites | Selected LEA The data will be verified and validated by the 
from statistically public websites Director of Student Privacy Policy in collaboration 
representative sample with the Department’ Chief Privacy Officer on at least 
reviewed for inclusion of a quarterly basis. Anomalous data will be noted in 
transparency best practices the comments field, and staff and contractors will be 
and compliance with legal questioned to ensure that district website reviews 
requirements relating to third are accurate and complete. No limitations are 
party contracting anticipated. The Chief Privacy Officer is responsible 
for certifying that the data are accurate. 
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Associated 
Strategic 
Objective 


Performance Metric 
(Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


3.3.A 


Number of sessions 
dedicated to improved data 
use provided to external 
grantees and stakeholders 
presented by Department 
employees or their 
contractors, or occurring at 
Departmental hosted events 


PPSS records 


The sessions will be tracked on a dashboard on 
PPSS'’s SharePoint site. Program offices submitting 
the data will be responsible for verifying its accuracy. 
The dashboard will include a statement that Program 
Offices submitting data must agree to ensure the 
accuracy of their information prior to entering it into 
the dashboard. OPEPD-PPSS would investigate 
further any anomalous data. Anomalous data would 
be researched using the original data submitted and 
follow-up calls to the program office will be made 
when necessary. One limitation of the collection 
could be Program Offices not capturing all qualifying 
sessions in the dashboard. The Department can use 
its multiple data focused teams, such as DST, 
EDFacts Data Governance Board, and InformED, to 
continuously remind Program Offices to enter their 
data. OPEPD-PPSS is responsible for certifying that 
the data are accurate to the best of their knowledge. 


3.3.B 


Number of newly added 
publicly available datasets in 
machine readable formats 


The 
Department’s 
Public Data 
Listing 


The Office of the Associate Commissioner for the 
Administrative Data Division in NCES will verify that 
the extract matches the inventory at the time of 
extraction. NCES and OPEPD-PPSS will discuss 
and review jointly and data identified as anomalous. 
Anomalous data would be researched using the 
original metadata submitted and the extraction. 
Follow-up calls to the contact person listed for the 
dataset in question will be made if necessary. The 
universe of the data used in this reporting does not 
currently reflect the full universe of data that are or 
could be made publicly available by the Department. 
There may also be access points to machine 
readable data not listed in the Public Data Listing. 
These gaps will continue to close as the Public Data 
Listing becomes more complete. Additionally, there 
may be a time in the future when the Public Data 
Listing is no longer the central access point for ED’s 
machine readable data. If this happens, we may 
need to revise the data source, but otherwise the 
metric will remain the same. OPEPD-PPSS is 
responsible for certifying that the data are accurate 
to the best of their knowledge. 


3.3.C 


Percentage of discretionary 
grant competitions that 
include the requirement to 
openly license to the public 
copyrightable grant 
deliverables created with 
Department grant funds 


Information 
Collection 
Packages 


OM-ICCD is responsible for ensuring the information 
is accurate, based on information received through 
the Information Collection Packages. If in reviewing 
the packages, OET has any questions about any 
information included in the packages, OET will first 
ask OM-ICCD. If OM-ICCD cannot find the answer 
they will facilitate communications with the 
appropriate Program Office to find the answer. The 
numerator and denominator are clearly defined, as is 
the process for calculating a percentage; therefore, 
there are no limitations or nuances that need to be 
documented at this time. The OET is responsible for 
certifying that the data are accurate to the best of 
their knowledge. 
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Strategic Goal 4: Strategic Goal 4: Reform the effectiveness, efficiency and accountability 
of the Department. 


Associated Performance Metric 
Strategic : Data Source(s) Data Verification and Validation Overview 
ce (Indicator) 
Objective 
4.1.A Number of evaluations to Report to OMB Division of Regulatory Services (DRS) will monitor 
identify potential EO 13771 the data and verify numbers. The Assistant General 
deregulatory actions that Counsel for Regulatory Services in OGC is 
included opportunity for responsible for certifying that the data are accurate. 
public input and/or peer 
review 
4.1.B Number of EO 13771 Department DRS will monitor the data and verify numbers. The 
deregulatory actions Records Assistant General Counsel for Regulatory Services 
recommended by the in OGC is responsible for certifying that the data 
Regulatory Reform Task are accurate. 
Force to the agency head, 
consistent with applicable 
law 
4.1.C Number of EO 13771 Federal Register | DRS will monitor the data and verify numbers. The 
deregulatory actions issued and ed.gov Assistant General Counsel for Regulatory Services 
that address in OGC is responsible for certifying that the data 
recommendations by the are accurate. 
Regulatory Reform Task 
Force 
4.1.D Number of EO 13771 Federal Register | DRS will monitor the data and verify numbers. The 
regulatory actions and, and ed.gov Assistant General Counsel for Regulatory Services 
separately, EO 13771 in OGC is responsible for certifying that the data 
deregulatory actions issued are accurate. 
4.1.E Total incremental cost of all Report to OMB 
EO 13771 regulatory actions 
and EO 13771 deregulatory 
actions (including costs or 
cost savings carried over 
from previous fiscal years) 
4.1.F Number of significant Reginfo.gov 
deregulatory actions 
submitted to OMB, the 
number of actions reflected 
on Reginfo.gov 
4.1.G Number of requests for E-mails between 
significance determination of | the Department 
deregulatory actions and OMB 
submitted to OMB, the 
number of such requests 
submitted to OMB by e-mail 
A2.A Percentage of key indicators | Office of the Each office that is responsible for a risk indicator 
that show improvement in Chief Financial within this composite metric maintains data 
performance and risk Officer will verification and validation procedures for that 
management calculate using indicator. Office of the Chief Financial Officer 
data from (OCFO) requires Sections | and III of this 
various sources | questionnaire be completed for all indicators and 
maintains copies as supporting documentation to 
help ensure data integrity. Judgement is required to 
define the list of risk indicators that comprise this 
composite metric, since it does not include all 
Strategic Plan metrics. OCFO leverages the Senior 
Management Council to decide on the final list of 
indicators that comprise this metric. OCFO is 
responsible for certifying that the data are accurate. 
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Associated 
Strategic 
Objective 


Performance Metric 
(Indicator) 


Data Source(s) 


Data Verification and Validation Overview 


A3.A 


44A 


Maturity level score of the 
Department's Information 
Technology (IT) Security 
Program and Practices as 
they relate to the Federal 
Information Security 
Modernization Act of 2014 
(FISMA) 


FEVS employee 
engagement index score 


Annual (FY) 
Inspector 
General FISMA 
Reporting 
Metrics Audit 
Report 


Office of 
Personnel 
Management 
(OPM) Federal 
Employee 
Viewpoint 
Survey (FEVS) 


As part of the annual audit process, the OIG 
disseminates draft reports for the Department to 
review and comment on its findings, 
recommendations, and assessments of maturity. 
OCIO leads coordination efforts across Department 
stakeholders and participates in review meetings 
with the OIG to verify and validate the information 
contained in the OIG’s report that provide the basis 
for its determination of the maturity levels. Each 
year the Office of Management and Budget 
publishes new FISMA guidance. OCIO reviews the 
new guidance and identifies areas that may impact 
how the Department's IT security program will be 
evaluated. OCIO leads coordination and 
communication efforts to ensure stakeholders and 
Department leadership understand the new 
guidance and any steps that need to be taken. The 
CIO is responsible for certifying that the data are 
accurate. 

Data verification and validation processes are 
integrated into the OPM FEVS survey results 
validation process. Any questionable FEVS results 
identified in annual reports would be brought to the 
attention of OPM. The Department would then work 
with the OPM point of contact to obtain clarity or 
resolve. Any questionable or anomalous FEVS data 
are brought to the attention of OPM. In turn, the 
OPM point of contact would research the matter 
and work with the Department to obtain clarity or 
resolve. There are no known data limitations. OPM 
is responsible for certifying that the data are 
accurate. 


4.4.B 


Percentage of positions with 
competencies identified 


Talent 
Management 
System (TMS) 


Data are processed and reviewed by LDD staff for 
accuracy. LDD staff will monitor progress of 
assessments. OM, OHR and LDD are responsible 
for certifying that the data are accurate. 


44.C 


Percentage of supervisors 
and managers with a 
performance plan critical 
element related to employee 
engagement 


TMS and USA 
Performance 


To ensure quality control, verification processors 
will self-check and cross check one another. The 
validation and affirmation of final numbers will be 
done by the Principal Operating Component (POC) 
EXO leadership to reconcile system reports. The 
Director of Workforce Relations Division will be 
responsible for certifying the data are accurate. 
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Appendix D: Programs by Goal 


Most of the Department’s programs are funded through discretionary appropriation acts enacted 
each fiscal year. However, there are many education programs—some of them large—that are 
funded directly through their authorizing statutes. For many budgeting purposes, these 
programs are classified as mandatory. For the purposes of this document, resources by goal 
are discretionary funds only. Mandatory programs that contribute to each goal are listed below. 


Strategic Goal 1 
SIU] 0) oLelame-jcclccmr-lalem(eler-lM-1ice)accm comin) elae-m(-r-laaliace meleicere)ai(-s me) mr-1| Made 


12 students in every community. 


Other discretionary Goal 1 programs/activities include the following: 


POC ACCT Goal # Objective # Program 

OESE ED 1 1.2,1.3, 1.4 Title | Grants to local educational agencies 

OESE ED L 1.2 State agency programs: Migrant 

OESE | ED 1 1.2 State agency programs: Neglected and 
delinquent 

OESE | ED 1 1.2 Special programs for migrant students: High 
School Equivalency Program 

OESE | IA 1 1.2 Impact Aid, Payments for federally connected 
children: Basic support payments 

OESE | IA 1.2 Impact Aid, Payments for federally connected 
children: Payments for children with disabilities 

OESE |IA A 12 Impact Aid, Facilities maintenance 

OESE IA 1 1.2 Impact Aid, Construction 

OESE | SIP 1 1.2,1.4 State assessments 

OESE | SIP 1 1.2 Education for homeless children and youth 
education 

OESE | SIP al 1.2 Training and advisory services 

OESE | SIP 1 1.2 Rural education 

OESE | SIP 1 1.2 Supplemental education grants 

OESE | SSCE al 1.2 School safety national activities 

OESE | IE 1 1.2 Indian Education: Grants to local educational 
agencies 

OESE | IE 1 1.2 Indian Education: Special programs for Indian 
children 

OESE |IE 1 1.2 Indian Education: National activities 

Oll 1&l il i a Opportunity Grants (proposed legislation) 

Oll I&l 1 1.3,1.4 Education innovation and research 

Oll I&l 1 1.1, 1.2 Charter schools grants 

Oll 1&l i Id dee Magnet schools assistance 

OESE/ | ELA 1 1.2 English language acquisition 

OLEA 

OSERS | SE 1 1.1,1.2,1.4 | Special education grants to States 

OSERS | SE 1 1.1, 1.2 Preschool grants 

OSERS | SE 1 14 Grants for infants and families 
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POC ACCT Goal # | Objective # Program 
OSERS | SE 1 1.1,1.2,1.4 | Technical assistance and dissemination 
OSERS | SE 1 1.1 Parent information centers 
OSERS | SE il 1.3 Education technology, media, and materials 


Mandatory programs supporting Goal 1 include: 
No additional programs. 


Strategic Goal 2 


=p-q oF: Valo Mm oloy-ja-y-Lexelaler-laVa-xeleter-ta(e)ar-l me) e)ele)acelalia(-t-mmlin]elge)v.-melelverelsi(-1-m ne) 
1Kox-3 (=) am -Lexe) ale)ailcome)e)ecelauvlalinvar-lalem elcelailelc-m-lamlaicelatirc-(emmaslelerelaiajelm-vare| 


el celelUCeqthVcmellir4-al aye 


Other discretionary Goal 2 programs/activities include the following: 


POC ACCT Goal # | Objective # Program 

OESE ED 2 2.1 Special programs for migrant students: College 
Assistance Migrant Program 

OSERS SE 2 2.2 State personnel development 

OSERS SE 2 2.2 Personnel preparation 

OSERS REHAB | 2 2.1,2.2 VR Training 

OSERS REHAB | 2 2.4, 202 Demonstration and training programs 

OSERS REHAB | 2 2.2 Independent living services for older blind 
individuals 

OSERS REHAB | 2 2.2 Helen Keller National Center for Deaf-Blind 
Youths and Adults 

OSERS APBH 2 2.1, 2.3 American Printing House for the Blind 

OSERS NTID 2 2.1, 2.2 National Technical Institute for the Deaf 

OSERS GU 2 2,1, 2.3 Gallaudet University 

OCTAE CTAE | 2 24,20 Career and technical education State grants 

OCTAE CTAE | 2 el 2A Career and technical national programs 

OCTAE CTAE 2 2.1 Adult basic and literacy education State grants 

OCTAE CTAE 2 2.1 Adult education national leadership activities 

FSA SFA 2 Bhs 2d Federal Pell grants: Discretionary 

FSA SFA 2 2,1, 2.3 Federal work-study 

OPE HE 2 2.1 Aid for institutional development: Strengthening 
tribally controlled colleges and universities 

OPE HE 2 mel Strengthening HBCUs 

OPE HE 2 2.1 Strengthening historically Black graduate 
institutions 

OPE HE 2 2.1 Strengthening HBCU masters program 

OPE HE 2 Pe a ae Minority science and engineering improvement 

OPE HE 2 2.1 Model transition programs for students with 
intellectual disabilities into higher education 

OCTAE HE 2 2.1 Tribally controlled postsecondary career and 
technical institutions 

OPE HE 2 2.1, 2.2, 2.3 | Federal TRIO programs 

OPE HE 2 2.1 Child care access means parents in school 
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POC ACCT Goal # Objective # Program 

OPE HE 2 2.1 Howard University: General support 

OPE HE 2 College Housing and Academic Facilities Loans 
Program Account: Federal administration 

OPE HE 2 Historically Black College and University Capital 
Financing Program Account: Federal 
administration 

OPE HE 2 Historically Black College and University Capital 
Financing Program Account: Loan subsidies 

FSA DM/ 2 2.4, 2.5 Student Aid Administration: Salaries and 

SAA expenses 
FSA DM/ 2 2.4, 2.4 Student Aid Administration: Servicing activities 
SAA 
Mandatory programs supporting Goal 2 include: 
POC ACCT Goal # Objective # Program 

OSERS REHAB | 2 2.1, 2.2, 2.3 Vocational rehabilitation, State grants 

OSERS REHAB | 2 2.1 Vocational rehabilitation, Grants to Indians 

FSA SFA 2 2.1, 2.3 Federal Pell grants: Mandatory 

FSA SFA 2 2.1, 2.3 Federal Pell grants: Mandatory funding for 
discretionary program costs 

FSA TEACH | 2 2.1, 2.3 TEACH Grants: New loan subsidy 

FSA FDSL 2 2.1, 2.3 Federal Direct Student Loans Program Account: 
New loan subsidy 

FSA FDSL 2 2.1, 2.3 Federal Direct Student Loans Program Account: 
New net loan subsidy (non-add) 

FSA FFEL 2 2.1 Federal Family Education Loans Program 
Account: Downward modification of existing 
loans (non-add) 

FSA FFEL 2 2.1 Federal Family Education Loans Program 
Account: Net modification of existing loans (non- 
add) 

FSA FFEL 2 2.1 Federal Family Education Loans Liquidating 
Account: Pre-1992 student loans 

FSA HEAL 2 2.1 Health Education Assistance Loans Liquidating 
Account 

OPE HE 2 2.1 Aid for institutional development: Mandatory 
strengthening tribally controlled colleges and 
universities 

OPE HE 2 2.1 Mandatory strengthening Alaska Native and 
Native Hawaiian-serving institutions 

OPE HE 2 2.1 Mandatory strengthening HBCUs 

OPE HE 2 2.1 Mandatory strengthening predominantly Black 
institutions 

OPE HE 2 2.1 Mandatory strengthening Asian American- and 
Native American Pacific Islander-serving 
institutions 

OPE HE 2 2.1 Mandatory strengthening Native American- 
serving nontribal institutions 
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POC ACCT Goal # | Objective # Program 

OPE HE 2 2.1, 2.2 Mandatory developing HSI STEM and articulation 
programs 

SFA 2 Zul Perkins loan repayments 

SFA FDSL | 2 2.1, 2.3 FDSL downward modification/negative loan 
subsidies 

SFA FFEL | 2 2.1 FFEL downward modification/negative loan 
subsidies 

SFA 2 2.1, 2.3 Student Financial Assistance debt collection 


Strategic Goal 3 


Sl icclaleitat-vamaar-meler-Uiiavam-Ceret-s-t-J le] [1 avamrclalemelcy-me)m-celUler-lalelamerclecMmialcolelera 
lol=) ancl amaarlar-le(-letlsd al mml alerg-r-Coi-10 my ola \Vr-\eavm eo) qelc-leqtlelat-w-lalemar-Var-yer-lac-daleaye 


Other discretionary Goal 3 programsi/activities include the following: 


POC ACCT Goal # | Objective # Program 
IES IES 3 3.1, 3.2, 3.3 | Research, development, and dissemination 
IES IES 3 3.1, 3.2, 3.3 | Statistics 
IES IES 2 3.1, 3.2, 3.3 | National assessment 
IES IES 3 3.1, 3.2, 3.3 | National Assessment Governing Board 
IES IES 3 cS, Research in special education 


Mandatory programs supporting Goal 3 include: 


No additional programs. 


Strategic Goal 4 
SCca Co) gta ealeMdat-celhV(-Jal-\-t-Mm ol niled(-lalenvar-lalem-Cexerel bl air-leliiiavme)maals 


Department. 


Other discretionary Goal 4 programs/activities include the following: 


POC ACCT Goal # Objective # Program 
ALL DM/PA | 4 Program Administration: Salaries and expenses 
ALL DM/PA | 4 Program Administration: Building Modernization 
OCR OCR 4 Office for Civil Rights 
OIG OIG 4 Office of Inspector General 


Mandatory programs supporting Goal 4 include: 


No additional programs. 
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Appendix E: Summary of Performance Evaluations 
Conducted During FY 2017 and Expected During FY 2018-19 


For a complete list of program evaluations and studies from the Office of Planning, Evaluation, 
and Policy Development, please visit 
http:/Awww2.ed.gov/about/offices/list/opepd/ppss/reports.html. For a complete list of evaluation 
studies of the National Center for Education Evaluation and Regional Assistance, please visit 


http://ies.ed.gov/ncee/projects/evaluation/index.asp. 


Evaluation Reports From FY 2017-18 
NEcNaCo) akc Ox=]a) (=) gm colam —elecersuicelam =AVccUlUr-lelelam-lalemac-lellelacclmatyyijc les 


Literacy 


An Exploration of Instructional Practices that Foster Language Development and 
Comprehension: Evidence from Prekindergarten through Grade 3 in Title | 
Schools 


Study Purpose: 

The National Assessment of Educational Progress (NAEP) found that one-third of fourth-grade 
students performed below a basic level in reading in 2011. Gaps in reading and language 
development between disadvantaged children and their more advantaged peers begin early, as 
documented by the Early Childhood Longitudinal Study. Closing the gap in language 
development and background knowledge is critical if children are to comprehend text because 
the research literature indicates that these areas of development are linked. Given the modest 
and inconsistent effects of large-scale early literacy interventions, the purpose of the Early 
Childhood Language Development study was to investigate types of instructional practices that 
hold potential promise for promoting children's language development and comprehension. 


Key Question(s) Addressed: 

e What classroom practices are associated with children's development of language skills, 
background knowledge, and comprehension in prekindergarten through 3rd grade in Title | 
schools? 


Design: 

This exploratory study included a sample of 83 Title | schools with prekindergarten programs to 
identify instructional practices associated with improved language development, background 
knowledge, and comprehension outcomes. In the 2011-12 school year, the study collected data 
for five grade cohorts (prekindergarten, kindergarten, first, second, and third grades) and 
classroom and student samples were selected for each. Data collection included a battery of 
student assessments and classroom observations. This study is exploratory, and its analyses 
estimated the associations between instructional practices and student outcomes to inform 
future rigorous evaluation of strategies to improve language and comprehension outcomes for 
disadvantaged children. 


Actual Completion Date: 
The final report was released in August 2017. 
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Key Findings: 

The exploratory analysis identified four instructional practices that show promise for improving 

young children’s language development and comprehension. The practices that were most 

consistently related to student growth include: 

e Engaging students in defining new words during or after reading a text 

e Helping students make connections between their prior Knowledge and the texts they read 

e Promoting higher-order thinking by asking questions that require students to analyze 
information, explain their thinking, and develop new ideas 

e Focusing students’ attention on the meaning of a text before reading it, such as by 
introducing the topic and encouraging predictions 


Link to Additional Information: http://ies.ed.gov/ncee/pubs/20174024 


Teacher Effectiveness 


Study of the Distribution of Effective Teaching 


Study Purpose: 

There is a persistent achievement gap in the United States where students from high-income 
families outperform those from low-income families on achievement tests. There is also 
substantial variation in the effectiveness of teachers. A key question for policy makers is 
whether policy initiatives focused on providing low-income students with equal access to 
effective teachers can address the achievement gap. This study provided information about the 
extent to which disadvantaged students received less-effective teaching than other students. 
The study also examined teacher mobility in participating districts and how patterns of mobility 
might contribute to unequal access. 


Key Question(s) Addressed: 

e Are low-income students taught by less effective teachers than high-income students? If so, 
to what extent would providing equal access to effective teachers reduce the student 
achievement gap? 

e Are there differences between high- and low-poverty schools in teacher hiring, transfer, and 
attrition? If so, are they consistent with inequitable access to effective teachers for low- 
income students? 


Design: 

The study documented low-income students’ access to effective teachers, as measured by 
value added across the 2008-09 through 2012-13 school years. The study also described 
district polices designed to address inequitable distribution of effective teaching implemented 
during those years. Lastly, the study examined teacher mobility patterns within participating 
districts. Annual data collection included district administrative records, such as student 
achievement to conduct value added analyses, as well as annual semi-structured interviews 
with district leadership to provide information on district policies. District personnel data were 
also collected to examine teacher mobility within participating districts. The study was 
conducted in 29 geographically dispersed school districts. 


Actual Completion Date: 
The final report on school years 2008—09 through 2012-13 was released in October 2016. 
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Key Findings from the Final Report: 

The final report focused on low-income students’ access to effective teachers and teacher 

mobility patterns in 26 districts and found that: 

e There are small differences in the effectiveness of teachers of high- and low-income 
students, on average. The average teacher of a low-income student is just below the 50th 
percentile of effectiveness based on value-added, while the average teacher of a high- 
income student is at the 51st percentile. Providing low-income students with equally 
effective teachers would not substantively reduce the achievement gap. 

e Inasubset of the study districts, there is meaningful inequity in teacher effectiveness in 
math. In 3 of the 26 study districts, providing low-income students with teachers whose 
effectiveness is equal to that of high-income students over a five-year period would reduce 
the math achievement gap by at least a tenth of a standard deviation of student 
achievement, the equivalent of about 4 percentile points. 

e Teacher hiring patterns are consistent with small inequities in access to effective teachers. 
High-poverty schools have more newly hired teachers than low-poverty schools, but this 
difference is likely to have a small influence on equity because (1) relatively few teachers 
are new hires (11 percent of teachers in high-poverty schools and 5 percent in low-poverty 
schools), and (2) performance of newly hired teachers improves quickly. On average, newly 
hired teachers become as effective as the average teacher after one year. 

e Teacher transfer patterns are also consistent with small inequities in access to effective 
teachers. Teachers who transfer to schools in a higher poverty category are less effective 
(43rd percentile) than the average district teacher. Teachers who transfer to schools ina 
lower poverty category are nearly as effective (48th percentile) as the average district 
teacher. These patterns likely have a small influence on equity since just under 4 percent of 
all teachers transfer across poverty categories each year. 

e Teacher attrition patterns do not contribute to inequity. Teachers who leave a district are 
less effective (44th percentile) than the average teacher, and more teachers leave high- 
poverty schools than low-poverty schools (10 percent versus 7 percent, respectively). 


Link to Additional Information: htips://ies.ed.gov/ncee/pubs/20174008/ 


Impact Evaluation of Teacher and Leader Evaluation Systems 


Study Purpose: 

The study is designed to examine the implementation of a package of performance evaluation 
system components and the impact of their use for formative purposes. These are components 
that states and districts might elect to include in their evaluation systems, with support from Title 
I| Part A funds under the Every Student Succeeds Act (ESSA). The components include 
measures of student achievement growth, classroom practices, and principal leadership. 


Key Question(s) Addressed: 

e What is the impact of the performance evaluation system on teachers’ classroom practices 
and student achievement? What are districts’ and educators’ experiences with 
implementation? 


Design: 

Within each of eight districts, approximately 15 elementary and middle schools were randomly 
assigned to receive the study’s measures of student achievement growth, classroom practices, 
and principal leadership during 2012-13 and 2013-14 (treatment group) or to participate only in 
the district’s usual performance evaluation system during the same time period. In treatment 
schools, each year, teachers received four rounds of classroom observations and feedback 
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sessions and information about their students’ achievement growth in math and/or reading using 
a statistical model known as value-added. Principals in treatment schools received feedback on 
their leadership practices twice per year. Study data collection included surveys of teachers and 
principals, observations of teachers’ classroom practices, and collection of student records data. 
The study also looks at the extent to which the measures were implemented as intended and 
whether the ratings from the measures reliably distinguish educator performance. 


Estimated or Actual Completion Date: 

A report on first year implementation findings was released in November 2016. A final report on 
impacts on educator practices and student achievement as well as implementation during the 
study’s second year is expected in fall 2017. 


Key Findings from the First Report: 

e The study’s performance measures were implemented generally as planned. Teachers and 
principals received multiple rounds of ratings and feedback on their practices. However, 
fewer principals and teachers accessed their student growth reports than the study 
intended. 

e Both classroom observation and student growth measures differentiated teacher 
performance, although observation scores were mostly at the upper end of the scale. 
Overall, observation scores varied across teachers, and both value-added scores and 
average classroom observation scores over the year had sufficient reliability to capture 
performance differences among some teachers. 

e The principal leadership measure differentiated performance, but principal self-ratings, 
teachers’ ratings of the principal, and the principal’s supervisor's ratings of the principal 
often differed. 

e Both teachers and principals in treatment schools reported receiving more feedback on their 
performance than did their counterparts. For example, teachers and principals in treatment 
schools reported spending more total time in performance feedback sessions across the 
year than teachers and principals in the control schools. 


Link to Additional Information: htips://ies.ed.gov/ncee/pubs/20174004/ 


Does Content-Focused Teacher Professional Development Work? Findings from 
Three Institute of Education Sciences (IES) Studies 


Study Purpose: 

Federal and local governments continue to invest billions of dollars each year in professional 
development for teachers. Until recently, there has been little rigorous evidence to inform the 
design and delivery of these professional development programs. Nevertheless, there has been 
growing consensus that deepening teachers’ content knowledge is an essential component of 
effective professional development in both reading and mathematics. Over the past decade, IES 
conducted three large-scale random assignment studies of teacher professional development in 
different grades in reading and math. These studies reveal a common pattern of findings on the 
impact of intensive, content-focused professional development on teaching and learning. The 
findings also highlight unresolved issues that future research might explore to advance our 
understanding and inform professional development policy and practice. 


Key Question(s) Addressed: 
e This brief synthesizes findings from three large-scale random assignment studies of 
professional development that were conducted by the National Center for Education 
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Evaluation and Regional Assistance in IES. Each study examined the impact of teacher 
professional development on teacher knowledge, practice, and student achievement. 


Design: 

Although the professional development programs in each study were different, they all 
emphasized building teachers’ content knowledge or knowledge about content-specific 
pedagogy. The programs combined summer institutes with periodic teacher meetings and 
coaching during the school year. These programs were compared to the substantially less 
intensive professional development that teachers typically received in study districts. The three 
studies included 270 second-grade reading teachers, 165 fourth-grade math teachers, and 195 
seventh-grade math teachers. 


Actual Completion Date: 
The brief was released in November 2016. 


Key Findings: 
e The professional development improved teachers’ knowledge and some aspects of their 
practice; 


e Improving teachers’ knowledge and practice did not lead to positive impacts on student 
achievement; 

e Most of the measured aspects of teachers’ knowledge and practice were not correlated with 
student achievement; and 

e The consistent pattern of findings suggests that future studies might seek to better 
understand on what aspects of teacher knowledge and practice professional development 
should focus, and how professional development can achieve a larger impact on knowledge 
and practice that also impacts student achievement. 


Link to Additional Information: http://ies.ed.gov/ncee/pubs/20174010/ 


School Choice 


Evaluation of the DC Opportunity Scholarship Program (OSP): Impacts One Year 
After Application 


Study Purpose: 

The April 2011 Scholarships and Opportunities for Results Act (SOAR Act) provided for a five- 
year continuation of a school choice program for low-income residents of Washington, DC. The 
program, still titled the Opportunity Scholarship Program (OSP), now provides annual 
scholarships of $8,000 (for grades K-8) or $12,000 (for grades 9-12), plus a cumulative inflation 
adjustment, to enable low-income students to attend private schools in DC in lieu of the public 
schools already available to them. The new law also mandated another independent, rigorous 
evaluation of the effectiveness of the program. 


Key Question(s) Addressed: 

e What is the impact of the OSP on student academic achievement and other measures of 
student success, overall and for subgroups of students identified in the statute as high 
priority? 

e What effect does the program have on student and parent perceptions of school safety and 
satisfaction, and on parents’ involvement in education of their children? 

e Does the program change students’ instructional environments and opportunities? 
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Design: 

The evaluation will primarily compare outcomes of approximately 1,800 student applicants 
randomly assigned by lottery to either receive a scholarship or not receive a scholarship. 
Lotteries of program applicants were conducted in spring 2012 (cohort 1), spring 2013 (cohort 
2), and spring 2014 (cohort 3). Data were collected for three follow-up years for each of the 
cohorts and, for students in both the scholarship and nonscholarship groups. The contractor 
administered academic assessments, and conducted student, parent, and principal surveys 
each spring (Spring 2013-17). 


Estimated or Actual Completion Date: 
The first impact report was released in spring 2017. The second and final impact reports are 
planned for early spring 2018 and late fall 2018. 


Key Findings from the First Impact Report: 

e The OSP had a statistically significant negative impact on mathematics achievement after 
one year. Math scores were lower for students who were offered or used OSP scholarships, 
compared to students that applied for but were not offered scholarships. There were no 
statistically significant impacts on reading scores for students in the evaluation overall. 

e The program did not have a statistically significant impact on parents’ or students’ general 
satisfaction with the school the child attended in that first year. The percentage of parents 
giving their child's school a grade of A or B was not statistically different when comparing 
parents of students who were offered or used OSP scholarships with the parents of students 
not selected for the scholarship offer. There were also no statistically significant differences 
when looking at student satisfaction with schools. 

e The program had a statistically significant positive impact on parents' perceptions of safety 
at the school their child attended in that first year. Parents of students who were offered or 
used OSP scholarships were more likely to indicate that their child's school was "very safe," 
compared with the parents of students not selected for the scholarship offer. Student 
perceptions of school safety were not significantly different between the groups. 

e The OSP did not have a statistically significant impact on parent involvement in education 
overall. However, for parents of students in grades 6—12, the program had statistically 
significant positive impacts on involvement in education-related activities and events at 
home after one year. 


Link to Additional Information: htips://ies.ed.gov/ncee/pubs/20174022/ 


Students with Disabilities 
National Longitudinal Transition Study (NLTS) 2012 


Study Purpose: 

Supporting youth’s successful transition to adult life is a priority under the Individuals with 
Disabilities Education Act (IDEA). IDEA provides federal funds to states and through them to 
school districts to assist in providing special education and related services. States and school 
districts must make a free appropriate public education (FAPE) available to all eligible children 
with disabilities. FAPE includes the provision of special education and related services provided 
in conformity with an individualized education program. IDEA requires that the first IEP to be in 
effect when a child turns 16 include postsecondary goals and transition services needed to help 
youth with disabilities complete high school prepared to achieve appropriate post-school 
outcomes, such as postsecondary education, jobs, and independent living. The NLTS 2012 is 
the third in a series examining the characteristics, school experiences, and post-high school 
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outcomes of a nationally representative sample of youth with disabilities. NLTS 2012 focuses on 
youth ages 13 to 21 (in December 2011), but also includes a small sample of students without 
disabilities to enable direct comparisons of students with and without an IEP. This study is part 
of the congressionally mandated National Assessment of the IDEA and is supported with funds 
authorized under Section 664 of IDEA. 


Key Question(s) Addressed: 

e How do the personal, family, and school characteristics and experiences of youth with 
disabilities differ from those of youth not served under IDEA? (Volume 1; March 2017) 

e How do the characteristics and experiences of youth vary across disability groups? (Volume 
2; March 2017) 

e How have the characteristics and experiences of youth with disabilities changed over time? 

e TO what extent do youth with disabilities make progress through high school compared to 
other youth? 

e Are youth with disabilities achieving the post-high school outcomes envisioned by IDEA, and 
how do their college, training, and employment rates compare to those of other youth? 

e How do these high school and postsecondary outcomes vary with student characteristics? 


Design: 

This descriptive study includes a nationally representative sample drawn from 432 school 
districts and special schools randomly sampled in 2011 and students randomly sampled within 
those districts. Survey data were collected in 2012-2013 from approximately 12,000 in-school 
youth and their parents, of which about 10,000 are students with IEPs representing each of the 
federal disability categories. The surveys asked about the background characteristics of 
secondary school youth and the schools they attend, their health, functional abilities, and 
engagement in school, the academic supports they receive, and their expectations for and steps 
to achieve transitions beyond high school. High school records and postsecondary enrollment 
information from the National Student Clearinghouse are being collected (from 2017-2019) to 
follow the sample of students through high school and beyond. This administrative data will be 
linked with the 2012-2013 survey data to examine high school course-taking and completion, 
and youth’s experiences with college, training, and employment. 


Actual Completion Date: 
The first two volumes of the study report describing survey results were released in March 2017. 


Key Findings (from the March 2017 volumes): 

e Youth with an IEP are more likely than their peers to be socioeconomically disadvantaged. 
Youth with an IEP are 12 percentage points more likely to live in low-income households 
and are less likely to have parents who are employed or have a college education. Among 
disability groups, youth with intellectual disability and youth with emotional disturbance are 
more socioeconomically disadvantaged and more likely to attend a lower-performing school 
than youth with an IEP overall. 

e The vast majority of youth with and without an IEP feel positive about school, but those with 
an IEP experience bullying and are suspended at higher rates. Like their peers, more than 
80 percent of youth in special education report that they are happy with school and with 
school staff. However, not only do youth with an IEP more commonly experience some 
types of bullying (e.g., being teased or called names) but, according to parent reports, they 
are more than twice as likely to be suspended or expelled from school. 

e Youth with an IEP are more likely than other youth to struggle academically, yet less likely to 
receive some forms of school-based support. Half of all youth with an IEP report they have 
trouble with their classes, about 15 percentage points more than reported by their peers. 
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However, they are less likely to report receiving school-based academic help before or after 
regular hours, although their parents more commonly help with homework and attend a 
parent-teacher conference. 

e Youth with an IEP lag their peers in planning and taking steps to obtain postsecondary 
education and jobs. Substantially fewer youth with an IEP expect to enroll in postsecondary 
education or training, compared to youth without an IEP. Reflecting these gaps, youth in 
special education are almost half as likely as their peers to report taking college entrance 
and placement tests. Forty percent report having recent paid work experience while in high 
school, compared with 50 percent of youth without an IEP. 

e Youth with autism, deaf-blindness, intellectual disability, multiple disabilities, and orthopedic 
impairments are most at-risk for not transitioning successfully beyond high school. Youth in 
these groups are less likely than all youth with an IEP to have key characteristics and 
experiences linked to success after high school, such as performing typical daily living tasks, 
engaging with friends and in school activities, or preparing for college, careers, and 
independent living. 


Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174016/ 


Pathways to Career or College 
Upward Bound at 50: Reporting on Implementation Practices Today 


Study Purpose: 

First established in 1965, Upward Bound is the oldest and largest of the federal college access 
programs targeted to low-income students and those who would represent the first-generation 
of college completers in their families. When the evaluation began in 2012, the program served 
more than 60,000 high school students at a cost of about $4,300 per youth. Upward Bound 
projects provide a wide array of academic and college transition support services, with seven 
types of services required by statute (“core” services). While much about the structure of 
Upward Bound and the services to be offered are prescribed in legislation, little is currently 
known about the intensity, duration, and mix of services provided by projects or about how they 
are delivered. Because of the importance of its mission, and the comprehensiveness and costs 
of its services, Upward Bound has long been of interest to policymakers. This report describes 
the approaches to providing program services, as reported by Upward Bound project directors, 
to help identify common (or uncommon but promising) practices that could inform program 
improvement studies that Congress requires the U.S. Department of Education to conduct. 


Key Question(s) Addressed: 

e Within the core service areas of the program, where do projects focus their efforts? 

e How are services delivered to Upward Bound participants? 

e In what ways does the focus or delivery of services vary across different Upward Bound 
projects? 


Design: 
This descriptive report relies primarily on findings from a summer 2013 survey of all regular 
Upward Bound project directors. 


Actual Completion Date: 
The report was released in November 2016. 
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Key Findings: 

e In four of the seven core service areas—coursework, tutoring, college exposure, and college 
application assistance—there was a dominant approach (used by at least 50 percent of 
projects) to how projects focused their activities. 

e When, where, and how services were delivered differed across service areas. For example, 
tutoring and college entrance exam preparation services were most commonly delivered in 
groups, while academic advising, college application assistance, and financial aid 
assistance were typically provided one-on-one. 

e Variation in the focus and delivery of services appears related to the urbanicity and type of 
institution (four-year, two-year, and non-higher education) that hosts the project but not to 
other project characteristics examined (e.g., project size, per-student funding, and whether 
the host institution was a Minority-Serving Institution). 


Link to Additional Information: http://ies.ed.gov/ncee/pubs/20174005/ 


Other 
Race to the Top: Implementation and Relationship to Student Outcomes 


Study Purpose: 

Beginning in 2010, Race to the Top (RTT) was a competitive grant program of the Department 
that funded general state competitions, state competitions focused on early learning, and district 
competitions focused on personalized learning. With funds from the Recovery Act, the general 
state competition awarded approximately $4 billion to states in support of comprehensive K-12 
education reform in several core areas, including teachers and leaders, standards and 
assessments, data systems, and school turnaround. This study examined the implementation of 
RTT and its relationship to student outcomes, focusing on the initial general state competition 
for RTT that began in 2010. 


Key Question(s) Addressed: 

e Which policies and practices promoted by the RTT program did RTT states report using, and 
how do they compare to the policies and practices that non-RTT states reported using? 

e Was receipt of an RTT grant related to improvement in student outcomes? 


Design: 

The RTT sample included all 50 states and DC. Data from interviews with all states and DC 
informed the first evaluation question. The second evaluation question was examined by 
comparing state-level trends in NAEP scores before and after the RTT competition, for states 
that were awarded an RTT grant to states that applied for but were not awarded an RTT grant. 


Actual Completion Date: 
The final report for RTT was released in October 2016. 


Key Findings from the Final Report: 

e In four of six areas examined, 2010 RTT grantees reported using more policies and 
practices promoted by RTT than states that did not receive a grant: The four areas with 
differences were standards and assessments, teachers and leaders, school turnaround, and 
charter schools. The other two areas were state capacity and data systems. 

e 2011 RTT grantees reported using more policies and practices promoted by RTT than 
states that did not receive a grant in one area, which was teachers and leaders. 
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e Across all states, use of RTT-promoted policies and practices were highest in the data 
systems area and lowest in the teachers and leaders’ area: states reported using 76 percent 
of the 8 RTT-promoted practices examined in data systems, but only 26 percent of the 39 
practices in teachers and leaders. 

e The relationship between RTT and student outcomes was not clear: Trends in student 
outcomes could be interpreted as providing evidence that RTT had a positive effect, a 
negative effect, or no effect. 


Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174001/ 


School Improvement Grants (SIG): Implementation and Effectiveness 


Study Purpose: 

In FY 2009, $3.5 billion ($546 million in regular appropriations and $3 billion from the Recovery 
Act) were allocated to the SIG program authorized by Title | of the ESEA. SIG intended to 
support rapid improvement in the nation’s persistently lowest-achieving schools, and funds were 
distributed to states by formula based on Title | allocations. States then competitively awarded 
funds to districts applying on behalf of their eligible schools. Schools receiving FY 2009 SIG 
funds were required over the three-year grant period to implement one of four prescriptive 
intervention models: turnaround, transformation, closure, or restart. This study examined the 
implementation and impacts of SIG, focusing on the first cohort of SIG schools implementing 
intervention models beginning in the 2010-11 school year. 


Key Question(s) Addressed: 

e Did SlG-funded schools use the improvement or turnaround strategies promoted by the four 
SIG intervention models, and how did they compare to strategies in schools not 
implementing a SIG-funded intervention model? How did states and districts support such 
efforts? 

e Did receipt of SIG funding to implement a school intervention model have an impact on 
outcomes for low-performing schools? 

e Was implementation of the four school intervention models related to improvement in 
outcomes for low-performing schools? 


Design: 

The SIG sample included about 500 schools in 60 districts from 22 states. This sample was 
purposely selected to support a regression discontinuity design to address the fourth evaluation 
question, exploiting cutoff rules that states used to identify their persistently lowest-achieving 
schools as eligible for SIG to implement one of the four intervention models. Data from state 
and district interviews, as well as school surveys from the SIG sample, were used to inform the 
third and fifth evaluation questions. Student- and school-level achievement data were also 
collected from administrative records up to the 2012-13 school year to inform the fourth and fifth 
evaluation questions. 


Actual Completion Date: 
The final report for SIG was released in January 2017. 


Key Findings from the Final Report: 

e SIG schools implementing one of the four models (transformation, turnaround, restart, or 
closure) reported using more practices than other schools: SIG schools reported using an 
average of 23 out of 35 practices, whereas other schools reported using 20 practices. 
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e Across all schools, use of SIG-promoted practices was highest in the area of comprehensive 
instructional reform strategies and lowest in the area of operational flexibility and support: 
Schools reported using 89 percent of the eight SIG-promoted practices examined in the 
comprehensive instructional reform strategies area, but only 43 percent of the two practices 
in operational flexibility and support (the other two areas examined were increasing teacher 
and principal effectiveness, and increasing learning time and creating community-oriented 
schools). 

e Implementing any of the four SIG models had no significant impacts on math or reading test 
scores, high school graduation, or college enrollment. 

e In elementary grades, student achievement gains did not differ across the four SIG models. 
In secondary grades, the turnaround model was associated with larger achievement gains 
than the transformation model. 


Link to Additional Information: htips://ies.ed.gov/ncee/pubs/20174013/ 


Implementation of Title | and Title Il-A Program Initiatives: Results from 2013-14 


Study Purpose: 

The Title | and Title II-A programs are a key part of the Elementary and Secondary Education 
Act (ESEA). These programs are intended to help provide all students with equal access to 
education by: providing financial assistance to schools and districts which have a high 
percentage of students from low-income families (Title 1) and improving teacher and principal 
quality (Title Il-A). Prior to the most recent reauthorization in December 2015, states were 
allowed to apply for waivers to certain No Child Left Behind Act requirements. A majority of 
states sought and received ESEA flexibility beginning in 2012. In exchange for waivers, they 
agreed to implement various reform principles, such as identifying and supporting schools with 
achievement gaps among student subgroups and implementing educator evaluation systems 
based on student achievement and multiple observations. 


The first report presents implementation data from states, districts, schools, and teachers prior 
to ESSA, under the No Child Left Behind Act and ESEA flexibility during the 2013-2014 school 
year. A subsequent report will look at implementation after reauthorization during the 2017-2018 
school year using data from states and districts. 


Key Question(s) Addressed: 

e What content standards and high school graduation requirements are states adopting, and 
what materials and resources are provided to support implementation? 

e What types of assessments do states and districts use, and what materials and resources 
are provided to support the implementation of assessments and use of assessment data? 

e What elements are included in states' accountability systems? How do states and districts 
identify and reward their highest-performing schools; how do they identify and support their 
lowest-performing schools; and how do they offer differentiated support for those schools 
that are neither highest-performing nor lowest-performing? 

e How do states and districts evaluate teacher and principal effectiveness and assess 
equitable distribution of teachers and principals, and what supports are provided to improve 
teacher and principal effectiveness? 

e How has student achievement changed over time? 


Design: 
Data were collected from all 50 states and the District of Columbia, a nationally representative 
sample of districts and schools, and teachers within those schools through surveys in the 2013— 
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2014 school year. These data informed the first four research questions. Extant math and 
reading achievement data from state standardized achievement tests and the NAEP were used 
to inform the fifth research question. 


Actual Completion Date: 
The report was released in January 2017. 


Key Findings from the First Report: 

e Most states adopted, and most principals and teachers reported implementing state 
standards that focused on college—and career-readiness. All but one state had committed to 
having college- and career-ready standards in place by 2013-14. A majority of principals 
(67-69 percent, depending on subject) reported fully implementing state content standards, 
and most teachers reported receiving professional development relevant to state content 
standards (79 percent of teachers) and weekly use of aligned instructional activities (92 
percent of teachers). 

e Many state assessments incorporated more sophisticated response formats to better assess 
students’ college- and career-readiness. In their reading/English language arts summative 
assessments, many states (24-36, depending on grade level) reported using extended 
constructed-response formats, a type of response format intended to assess higher-order 
thinking skills. Nineteen states used this response format in math assessments. 

e States used ESEA flexibility to reset their accountability goals and to target a narrower set of 
schools for additional support. Forty-three states had received ESEA Flexibility for the 
2013-14 school year. The most common accountability goal adopted by states with ESEA 
Flexibility (28 of the 43 states) was reducing by half the percentage of students and 
subgroups not proficient in 6-8 years. States with ESEA Flexibility identified 5 percent of 
Title | schools as lowest performing and an additional 10 percent of Title | schools with 
substantial student achievement gaps, compared to non-Flexibility states that reported 
identifying 43 percent of Title | schools as lowest performing. 

e Almost all states adopted new laws or regulations related to educator evaluation systems 
between 2009 and 2014, and most districts reported full or partial implementation in 2013— 
14. Only four states had not adopted new teacher evaluation laws or regulations by 2014, 
and a majority (59 percent) of districts reported fully implementing, piloting, or partially 
implementing a new teacher evaluation system. However, few districts (18 percent) reported 
using evaluation system measures of student achievement growth and classroom practice 
consistent with emerging research. 

e Proficiency rates on the NAEP slightly increased from 2005 to 2015, with the largest 
increases in 4th and 8th grades and smaller or no increases in 12th grade. Overall 
proficiency rates increased by statistically significant levels of 4-5 percentage points in 4th 
and 8th grade reading and math and by 2 percentage points in 12th grade reading. 
Statistically significant increases in proficiency were also evident for economically 
disadvantaged students in both subjects and across all three grades (by 4—7 percentage 
points), and in the large majority of individual states (46—51 states, depending on grade and 
subject). 


Link to Additional Information: https://ies.ed.gov/ncee/pubs/20174014/ 


Evaluation of NAEP Achievement Levels 


Study Purpose: 
Under the provisions of P.L. 107-279, the Secretary is required to provide for continuing review 
of the NAEP assessment. The legislation identifies the issues to be addressed in the reviews, 
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one of which includes the requirement to evaluate whether the NAEP achievement levels, 
established by the National Assessment Governing Board, are “reasonable, valid, reliable and 
informative to the public.” Section 303(e)(2)(C) of the Education Sciences Reform Act of 2002 
(P.L. 107-279) states that NAEP achievement levels shall be used on a trial basis until the 
Commissioner of the National Center for Education Statistics (NCES) determines, as a result of 
an evaluation under Sec. 303(f), that such levels are “reasonable, valid, and informative to the 
public.” This independent evaluation was intended to provide IES and the Commissioner of 
NCES with information necessary to inform the decision about whether the current trial status of 
the NAEP achievement levels can be removed or whether they should remain in trial status. 


Key Objectives: 

e Determine how “reasonable, valid, reliable and informative to the public” will be 
operationalized in this study. 

e Identify the kinds of objective data and research findings that will be examined. 

e Review and analyze extant information related to the study’s purpose. 

e Gather other objective information from relevant experts and stakeholders, without creating 
burden for the public through new, large-scale data collection. 

e Organize, summarize, and present the findings from the evaluation in a written report, 
including a summary that is accessible for nontechnical audiences, discussing the 
strengths/weaknesses and gaps in knowledge in relation to the evaluation criteria. 

e Provide, prior to release of the study report, for an independent external review of that report 
for comprehensiveness, objectivity, and freedom from bias. 


Design: 

This study focused on the achievement levels used in reporting NAEP results for the reading 
and mathematics assessments in grades 4, 8, and 12. Specifically, the study reviewed 
developments over the past decade in the ways achievement levels for NAEP are set and used 
and will evaluate whether the resulting achievement levels are “reasonable, valid, reliable, and 
informative to the public.” The study relied on an independent committee of experts with a broad 
range of expertise related to assessment, statistics, social science, and education policy. The 
project is receiving oversight from the Board on Testing and Assessment and the Committee on 
National Statistics of the National Research Council. 


Actual Completion Date: 

The report from this study was released final form in April 2017 and may be downloaded as a 
PDF or read online at https://www.nap.edu/catalog/23409/evaluation-of-the-achievement-levels- 
for-mathematics-and-reading-on-the-national-assessment-of-educational-progress. 


Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/other_naep.asp 


Policy and Program Studies Service 


A Study of Practices to Improve Learning in Kindergarten Through Third Grade 


Study Purpose: 

This study summarized how (a) aligning preschool through third grade education and (b) 
differentiating instruction for children in kindergarten and first grade may build on the positive 
effects of preschool and help students in kindergarten through third grade (K—3) make cognitive, 
social-emotional, and academic gains. In addition, the study conducted case studies of five 
programs that may build on the positive effects of preschool by using policies, programs, and 
practices for alignment and differentiation. 
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Key Question(s) Addressed in the Literature Review: 

e What approaches does the research and theoretical literature suggest for aligning preschool 
through third grade (P—3) education, and what is the quality of the research studies? 

e What are the findings from studies of differentiated instruction on children in kindergarten 
and first grade, and what is the quality of these studies? 


Key Question(s) Addressed in the Case Studies: 
e What approaches did the five programs use to implement P—3 alignment? 
e In programs that implemented differentiated instruction, what approaches did staff use? 


Design: 

The literature review covered two topics: (1) preschool and K—3 alignment, and (2) differentiated 
instruction in kindergarten and first grade. The case studies examined five sites that 
implemented P—3 alignment or differentiating instruction in kindergarten and first grade. The 
research team interviewed principals, teachers, evaluators, and funders to understand 
programs’ characteristics, challenges and solutions, and the sustainability of the programs. 


Actual Completion Date: 
The literature review was released in August 2016, and the case study report was released in 
December 2016. 


Key Findings from the Literature Review: 

e Nearly all qualitative studies and policy and theory articles on P—3 alignment suggest 
aligning standards, curriculum, instruction, assessments, and environments across 
preschool and grades K-3. 

e P-3 alignment could be supported by establishing similar teacher education and training 
requirements and developing longitudinal data systems that integrate preschool and 
K-12 data. 

e Of the 17 quantitative studies of differentiated instruction, one Randomized Control Trial of 
the Individualized Student Instruction With Assessment to Instruction intervention 
demonstrated positive results on reading outcomes and had the potential to meet the criteria 
for strong causal evidence. 

e Qualitative studies of differentiated instruction indicate that opportunities for peer 
collaboration and guidance by mentors may be helpful to improve teacher practice related to 
differentiation. 

e Key findings from the case studies: 

o All five programs aligned instruction across grades by coordinating standards, curricula, 
instructional practices and professional development. 

o Common elements of these programs included the use of professional learning 
communities, coaches, parent engagement, and play-based or student-initiated 
learning. 

o All five programs reported using strategies to accommodate students’ different skill 
levels, including modifying assignments, adapting learning materials, providing different 
levels of support, or using small-group instruction. 


Links to Additional Information: 


e Literature review: http://www2.ed.gov/rschstat/eval/disadv/p-3-alignment-differentiated- 
instruction/report. pdf. 
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e Case studies: http://www2.ed.gov/rschstat/eval/implementing-early-strategies/report.pdf. 


Study of Emerging Teacher Evaluation Systems 


Study Purpose: 

This study will provide descriptive information on the design and early implementation of teacher 
evaluation systems in eight school districts. The findings are intended to help other districts and 
states learn from the experiences of eight districts featured in the study, and apply the design 
and implementation lessons to their own work as it relates to teacher evaluation and support. 


Key Question(s) Addressed: 

e What key priorities and measures informed the design of the new teacher evaluation 
systems? 

e What steps did the districts take prior to full implementation to test the system and prepare 
teachers and staff to implement it? 

e How did the districts structure and conduct the classroom observation component of their 
teacher evaluation systems? 

e How did the districts analyze student performance and other data to evaluate teacher 
performance? 

e How did the districts use, or plan to use, teacher evaluation results to make personnel 
decisions? To what extent were professional development and career advancement 
decisions tied to evaluation results? 

e What administrative structures did districts use to support their new teacher evaluation 
systems? 

e What are the perceived early effects of the teacher evaluation systems on the professional 
practices of teachers, principals, and district administrators? 


Design: 

This descriptive study relied on interviews with key district administrators, principals, teachers, 
and representatives of community stakeholder groups, from eight districts, who were involved in 
the development and early implementation of the respective districts’ teacher evaluation system. 
Given the limited sample, the findings cannot be generalized to other districts. 


Actual Completion Date: 
The final report was released in November 2016. 


Key Findings: 

e Teachers and central office staff generally agreed that the foremost goal of the teacher 
evaluation system was to improve instruction. 

e Teacher and principal input during the design and/or pilot test phase strongly influenced 
decisions regarding system modifications in six districts, according to district administrators. 

e Classroom observations varied in frequency, duration, and degree of formality in all eight 
districts. In addition, principals reported challenges in finding time to conduct teacher 
observations. 

e Six districts used multiple approaches for measuring teacher impact on student 
performance, including individual and/or school-level value-added models. 

e Districts used teacher evaluation results for a range of purposes, including targeted 
professional development and support, career ladders and performance pay, and in some 
instances, redeployment or release of teachers identified as ineffective. 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 131 


APPENDICES 


e The majority of districts created relatively simple, streamlined structures to administer their 
teacher evaluation systems. 

e Teachers reported that they believed that the classroom observations and feedback helped 
them become better teachers. 


Link to Additional Information: 
http:/Awww2.ed.gov/about/offices/list/opepd/ppss/reports.html#tq. 


State Efforts to Promote Equitable Access to Effective Teachers 


Study Purpose: 

This report provides a broad overview of state efforts, as of the 2011—12 school year, to monitor 
equitable access to qualified and effective teachers among schools; develop and adopt multiple 
measures of teacher performance to rate teachers among at least three performance levels; 
and implement targeted strategies for promoting equitable access to qualified and effective 
teachers in schools serving high proportions of poor and/or minority students. The report 
examines the use of measures of teacher qualifications and teacher performance in the 
implementation of these strategies. 


Key Question(s) Addressed: 

e What measures did states use to monitor equitable access to qualified and effective 
teachers among schools? 

e To what extent were states developing or using multiple measures of teacher performance 
to rate teachers among at least three performance levels? 

e What strategies did states use to promote equitable access to qualified and effective 
teachers in schools serving high proportions of poor and/or minority students? 


Design: 

This report is based on telephone interviews with officials in SEAs in all states, the District of 
Columbia, and Puerto Rico. Study staff also reviewed extant sources including equity plans, 
Consolidated State Performance Reports, reports from federal monitoring visits, and other 
information on state websites. The interviews were conducted between August 2011 and 
January 2012, and the study reflects state efforts that were underway at that time, and predates 
implementation of the fall 2011 ESEA Flexibility initiative as well as the recent state equity plans 
submitted under the Excellent Educators for All initiative. 


Actual Completion Date: 
The final report was released in January 2017. 


Key Findings: 

e In 2011-12, states most commonly monitored equitable access to qualified and effective 
teachers among schools using measures of teacher qualifications. 

e Four states reported using measures of teacher performance—student achievement growth 
and/or measures of teacher practice—to monitor equitable access to qualified and effective 
teachers among schools, and nine states used teacher performance measures to monitor 
the quality of the teacher workforce overall. 

e In two of the four states that reported using teacher performance measures to monitor 
equitable access among schools, officials reported seeing larger inequities than were 
previously detected using measures of teacher qualifications alone. 
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e Six states reported that they had adopted multiple measures of teacher performance and 
were using them to rate teachers among at least three performance levels in 2011-12, and 
38 states indicated that there were in the process of developing such measures. 

e Offering monetary incentives was the most common strategy that states reported using in 
disadvantaged schools to promote equitable access to qualified and effective teachers 
among schools (24 states). 

e Other state-reported strategies that were directed specifically at disadvantaged schools for 
promoting equitable access were specialized professional development (14 states) and 
teacher recruitment and preparation programs (14 states). 


Link to Additional Information: 
http://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#tq. 


Feasibility Study on Improving the Quality of School-Level Expenditure Data 


Study Purpose: 

This study explores the feasibility of improving the collection of school-level expenditure data by 
examining the nature and quality of school-level fiscal data collection in five states and four 
school districts that had developed their own systems for collecting and reporting school-level 
expenditures: Florida, Hawaii, Ohio, Rhode Island, Texas, Baltimore City, Hillsborough County, 
Houston, and Los Angeles. 


Key Question(s) Addressed: 

e Instates and districts that have developed systems to report expenditures at the school 
level, what types of personnel and nonpersonnel expenditures are included in the school- 
level data? 

e To what extent do the sites track actual expenditures to individual schools versus allocating 
or prorating expenditures to schools using formulas? 

e How consistent are school-level expenditure data obtained from these systems with similar 
data from other sources? How do the funding amounts attributed to individual schools based 
on formula allocations compare to those based on tracking actual expenditures? 

e What lessons can other states and districts learn from these sites if they wish to implement 
systems for reporting accurate and reliable data on school-level expenditures? 


Design: 

The study included (1) surveys and interviews of officials to understand the process of collecting 
school-level expenditure data and (2) collection and analysis of school-level spending data to 
examine data quality issues. The study examined three aspects of data quality: the 
comprehensiveness of school-level spending data, consistency with other data sources, and the 
relative accuracy of allocating expenditures to schools by formula (rather than tracking actual 
expenditures for each school). 


Actual Completion Date: 
The final report was released in January 2017. 


Key Findings: 

e Study sites reported that they attributed most categories of spending to the school level, 
including salaries for teachers, administrators, and other support staff as well as 
nonpersonnel items, such as textbooks, instructional materials, furniture and equipment, and 
computers and software. 
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e The school districts and states in this study attributed an average of three-quarters of 
operational expenditures to individual schools, demonstrating that it is feasible to link a 
significant share of spending to the school level. 

e Most of the expenditures that the study sites attributed to schools were directly tracked to 
schools (85 percent) rather than simply being allocated by formula (8 percent). 

e Comparisons between the site-reported school-level expenditures and other data sources 
showed a relatively high degree of consistency for salary expenditures, but nonpersonnel 
expenditure data were much less consistent. 

e Allocating expenditures to schools by formula (e.g., based on total salaries or staff) 
appeared relatively accurate for health benefits and less accurate for pension benefits, pupil 
support staff, and instructional support staff. 

e = Instituting a system for collecting school-level expenditure data typically required new 
hardware and software (eight sites), changes to charts of accounts (six sites), and staff 
training (eight sites). 

e Advice that interviewees offered for others aiming to implement school-level expenditure 
data systems was to get stakeholders involved, communicate clearly and frequently, and 
think long-term about future data needs. 


Link to Additional Information: 
https://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#title. 


Study of Experiences and Needs of Rural Education Achievement Program 
(REAP) Grantees and Subgrantees 


Study Purpose: This descriptive study examined how grantees and subgrantees use REAP 
funds provided through the Small, Rural School Achievement (SRSA) and Rural and Low- 
Income Schools (RLIS) programs—on their own or in combination with other federal funds—as 
well as to explore technical assistance needs related to REAP. 


Key Question(s) Addressed: 

e What role do states play in supporting the Department's SRSA and RLIS eligibility and 
award determination process? 

e How do districts use their SRSA or RLIS funds? 

e To what extent do SRSA-eligible districts use REAP Flex and for what purposes? 

e What recommendations do states and districts have for improving the operation of the 
SRSA and RLIS programs? 


Design: 

The study consisted of: 1) a survey of a sample of approximately 1,000 SRSA grantees and 
RLIS subgrantee districts; 2) telephone interviews with a sample of 30 SRSA grantees and RLIS 
subgrantees; and 3) telephone interviews with REAP coordinators in all states receiving REAP 
funds. Data collection began in winter 2015 and was completed by spring 2016. At the state 
level, the study included interviews with all state REAP coordinators about state goals and 
priorities, the planning process for use of RLIS funds, the eligibility process for districts, 
management and distribution of SRSA and RLIS funds, and recommendations for the program. 
At the school district level, the study included an online survey of a nationally representative 
sample of REAP coordinators about the REAP eligibility determination process, administration 
of REAP, and challenges and technical assistance needs related to REAP, as well as telephone 
interviews with a subsample of 30 districts. The interviews included questions about program 
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administration, technical assistance needs, and recommendations for changing the REAP 
program to better meet the needs of rural districts. 


Actual Completion Date: 
The report was completed in December 2016. 


Key Findings: 

e States supported the Department in determining REAP eligibility by providing district-level 
data and reviewing the accuracy of Department-provided data. 

e All 43 states with RLIS-eligible districts chose to make subgrants to districts on the basis of 
a funding formula rather than on a competitive basis, and 28 of these states based the 
subgrant amount entirely on average daily attendance. 

e Districts most frequently used SRSA and RLIS funds to improve or expand access to 
technology (71 percent of SRSA districts and 71 percent of RLIS districts) and to provide 
educator professional development (45 percent of SRSA districts and 58 percent of RLIS 
districts). 

e Forty-six percent of SRSA district coordinators reported exercising REAP-Flex; of these, 82 
percent reported that they used funds eligible for REAP-Flex to maintain a stable level of 
funding for ongoing activities. 

e The majority of both district and state REAP coordinators were highly satisfied with REAP as 
a whole. However, they provided recommendations for improvement to REAP in three 
categories: (1) improved timelines for eligibility and award determination, (2) more 
information on allowable uses of funds and REAP-Flex, and (3) revised eligibility criteria. 


Link to Additional Information: 
https://www2.ed.gov/about/offices/list/opepd/ppss/reports.html#rural. 


Summary of Performance Evaluations Expected During FY 
2018 and FY 2019 
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Teacher Effectiveness 
Impact Evaluation of Teacher and Leader Evaluation Systems 


Study Purpose: 

The study is designed to examine the implementation of a package of performance evaluation 
system components and the impact of their use for formative purposes. These are components 
that states and districts might elect to include in their evaluation systems, with support from Title 
I| Part A funds under the ESSA. The components include measures of student achievement 
growth, classroom practices, and principal leadership. 


Key Question(s) Addressed: 

e What is the impact of the performance evaluation system on teachers’ classroom practices 
and on student achievement? 

e What are districts’ and educators’ experiences with implementation? 


Design: 
Within each of eight districts, approximately 15 elementary and middle schools were randomly 
assigned to receive the study’s measures of student achievement growth, classroom practices, 
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and principal leadership during 2012—13 and 2013-14 (treatment group) or to participate only in 
the district's usual performance evaluation system during the same time period. In treatment 
schools, each year, teachers received four rounds of classroom observations and feedback 
sessions and information about their students’ achievement growth in math and/or reading using 
a statistical model known as value-added. Principals in treatment schools received feedback on 
their leadership practices twice per year. Study data collection included surveys of teachers and 
principals, observations of teachers’ classroom practices, and collection of student records data. 
The study also looks at the extent to which the measures were implemented as intended and 
whether the ratings from the measures reliably distinguish educator performance. 


Estimated or Actual Completion Date: 

A report on first year implementation findings was released in November 2016. A final report on 
impacts on educator practices and student achievement as well as implementation during the 
study’s second year was released in December 2017. 


Key Findings from the First Report: 

e The study’s performance measures were implemented generally as planned. Teachers and 
principals received multiple rounds of ratings and feedback on their practices. However, 
fewer principals and teachers accessed their student growth reports than the study 
intended. 

e Both classroom observation and student growth measures differentiated teacher 
performance, although observation scores were mostly at the upper end of the scale. 
Overall, observation scores varied across teachers, and both value-added scores and 
average classroom observation scores over the year had sufficient reliability to capture 
performance differences among some teachers. 

e The principal leadership measure differentiated performance, but principal self-ratings, 
teachers’ ratings of the principal, and the principal’s supervisor's ratings of the principal 
often differed. 

e Both teachers and principals in treatment schools reported receiving more feedback on their 
performance than did their counterparts. For example, teachers and principals in treatment 
schools reported spending more total time in performance feedback sessions across the 
year than teachers and principals in the control schools. 


Link to Additional Information: 
https://ies.ed.gov/ncee/projects/evaluation/tq_performance.asp. 


Impact Evaluation of the TIF 


Study Purpose: 

The purpose of the TIF program is to develop and implement performance-based compensation 
systems (PBCSs) for teachers, principals, and other personnel in high-need schools. Research 
indicates that high-quality teachers are critical to raising student achievement in low-performing 
schools, but schools most in need often have difficulty in attracting and retaining high-quality 
teachers. Performance pay is a policy promoted by the TIF program to improve the quality of 
teachers in high-need schools. This evaluation studies performance pay that provides 
substantial and differentiated bonus pay to high-performing teachers in low-performing schools 
with high-need students. 


Key Question(s) Addressed: 


e What are the characteristics of all TIF grantee districts and their PBCSs? What 
implementation experiences and challenges did TIF districts encounter? 
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e How do teachers and principals in schools that did or did not offer pay-for-performance 
bonuses compare on key dimensions, including their understanding of TIF program 
features, exposure to TIF activities, allocation of time, and attitudes toward teaching and the 
TIF program? 

e What is the impact of pay-for-performance bonuses on students’ achievement on state 
assessments of math and reading? 

e How do pay-for-performance bonuses affect educator mobility, including whether mobility 
differs by educator effectiveness? 


Design: 

Study schools were randomly assigned within a grant to either implement all components of the 
PBCS or the PBCS with a 1 percent across-the-board bonus in place of the differentiated 
effectiveness incentive component of the PBCS. Data collection included a grantee survey, a 
survey of teachers and principals, teacher and principal school assignment records, student 
record information (Such as student demographics and student test scores), and grantee 
interviews. 


Estimated or Actual Completion Date: 

The first report was released in September 2014. The second report was released in September 
2015. The third report was released in August 2016. The fourth and final report, which updated 
implementation information and impacts, was released in December 2017. 


Key Findings from the Third Report: 

The main findings among all TIF districts with 2010 awards are: 

e Similar to the previous two years, most districts (88 percent) implemented at least three of 
the four required program components for teachers. 

e By the third year, reported implementation challenges decreased with no more than one-fifth 
of TIF districts reporting any major challenges. 

e For the subset of 10 districts that agreed to participate in a random assignment study, key 
findings on the effect of pay-for-performance on educators include the following: 

e After three years of TIF implementation, average student achievement remained 1 to 
2 percentile points higher in schools that offered pay-for-performance bonuses than in 
schools that did not. This difference was equivalent to a gain of about four additional weeks 
of learning. 

e Atleast half of the evaluation districts each year met the grant guidance for awarding 
differentiated performance bonuses for teachers. However, in each year, no more than 2 of 
the 10 districts awarded bonuses for teachers that were substantial or challenging to earn. 

e Teachers’ understanding of performance measures continued to improve, but only about 60 
percent of teachers correctly reported being eligible for a performance bonus. In addition, 
teachers believed that the maximum bonus they could earn was no more than two-fifths the 
size of the actual maximum bonus that districts awarded, a finding similar to previous years. 


Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tg_incentive.asp. 


Impact Evaluation of Support for Principals 


Study Purpose: 

Title Il, Part A, the Improving Teacher State Formula Grants program, is the primary federal 
funding under ESEA to support high-quality educators. The program targets high-poverty 
districts and funds a broad array of allowable activities for principals and teachers, such as 
support for certification, teacher mentoring and induction, intensive professional development, 
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recruitment, retention, and merit-based teacher and principal pay strategies as well as class 
size reduction. Principals, through a collective focus on instructional and organizational 
leadership and human capital management, have the potential to greatly influence the quality of 
instruction. However, there is limited evidence about the effectiveness of principal professional 
development programs and their ability to improve principals’ leadership skills and school 
quality. This evaluation studies professional development for principals and thus provides an 
important source of information for this program. 


Key Question(s) Addressed: 

e What are the professional development experiences of principals? 

e What are the initial impacts on school climate and educator behaviors of providing principals 
structured and intensive professional development? 

e What are the impacts on teacher retention, the effectiveness of instructional staff, and 
student achievement of providing principals with structured and intensive professional 
development? 


Design: 

The study team randomly assigned within 10 districts a total of 100 elementary schools to a 
treatment or control group. Treatment group principals will be offered intensive professional 
development provided by the University of Washington’s Center for Educational Leadership 
during the 2015-16 and 2016-17 school years. The Center for Educational Leadership was 
competitively selected to provide the professional development focused on in this study. The 
professional development includes a heavy emphasis on instructional leadership activities, 
including support in conducting school walkthroughs and classroom observations with 
constructive feedback to facilitate teacher growth focused on improving student achievement. 
Control group principals received supports normally offered by the district. Data collection 
included: information about the professional development delivered and experienced by the 
participating principals; teacher and principal surveys and periodic logs of principal daily 
activities to document intermediate outcomes, such as principal behaviors and school climate; 
and administrative records to document student outcomes (e.g., achievement, behavior, 
attendance) and teacher outcomes (e.g., retention of effective teachers, quality of newly hired 
teachers). 
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Estimated or Actual Completion Date: 
The report, which will focus on implementation and intermediate outcomes over two years 
(2015-16 and 2016-17), is expected in spring 2019. 


Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tq_principals.asp. 


Study of Teacher Preparation Experiences and Early Teacher Effectiveness 


Study Purpose: 

A primary focus of the ESSA’s Title II Part A is on the improvement of teacher quality. Little 
research exists, however, to inform how best to prepare teachers for the classroom. This study 
provides descriptive information on the preparation experiences of a large sample of novice 
teachers. It also will examine whether the instructional skills that teachers learn about and have 
opportunities to practice in their preparation programs are associated with teachers’ 
effectiveness once they are in the classroom. 


Key Question(s) Addressed: 

e What experiences do novice teachers report receiving as part of their preparation program? 

e What are the relationships between teacher-reported preparation experiences and novice 
teachers’ effectiveness in improving student achievement? 


Design: 

Approximately 3,200 novice language arts and/or math teachers from grades 4 through 6 
participated in the study. In spring 2015, the teachers responded to a survey focused on their 
preparation experiences related to instructional skills that have been shown to be associated 
with teacher value-added scores. For each instructional skill, the teachers were asked about the 
nature of their learning experiences, including how they learned (e.g., through coursework or 
through practice in K-12 classroom) and the extent to which they found the experience useful. 
The study will also compute value-added scores for teachers, based on students’ state math 
and English language arts tests, and examine the relationships between teacher preparation 
experiences and teacher value-added scores. 


Estimated or Actual Completion Date: 
A report describing teachers’ preparation experiences and the relationship between experiences 
and teacher value-added scores is expected by fall 2018. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/tq_teacherprep early.asp. 


Impact Evaluation of Professional Development for Teachers on Data-Driven 
Instruction 


Study Purpose: 

Title Il, Part A, the Improving Teacher Quality State Formula Grants program, is the primary 
federal funding under the ESEA to improve teacher quality. The program targets high-poverty 
districts and funds a broad array of allowable activities including professional development for 
teachers. Teachers and schools have access to ongoing data from student performance and 
assessments that provide information about students' academic needs. This study evaluates the 
effectiveness of professional development for teachers to use such data to guide their 
classroom instruction. 
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Key Question(s) Addressed: 

e What are the effects on student achievement of providing professional development to 
principals and teachers that focuses on using individual student academic information 
already available to school staff to guide instruction? 

e What are the effects of providing this professional development on teachers’ and principals’ 
use of individual student academic information to inform instructional support, planning, and 
practice? 


Design: 

The study is a randomized controlled trial with a sample of 104 schools in 12 districts. In each 
district, schools were randomly assigned to treatment (professional development) or to control 
(business as usual) status. Data collection consists of a teacher survey, a principal survey, and 
student administrative records, including student state standardized achievement test scores. 


Estimated or Actual Completion Date: 
The report is expected in summer 2018. 


Link to Additional Information: http://ies.ed.gov/ncee/projects/evaluation/tq_datadriven.asp. 


School Choice 


Evaluation of the DC Opportunity Scholarship Program: Impacts Two Years After 
Application and Final Report 


Study Purpose: 

The April 2011 SOAR Act provided for a five-year continuation of a school choice program for 
low-income residents of Washington, DC. The program, still titled the Opportunity Scholarship 
Program (OSP), now provides annual scholarships of $8,000 (for grades K—8) or $12,000 (for 
grades 9-12), plus a cumulative inflation adjustment, to enable low-income students to attend 
private schools in DC in lieu of the public schools already available to them. The new law also 
mandated another independent, rigorous evaluation of the effectiveness of the program. 


Key Question(s) Addressed: 

e What is the impact of the OSP on student academic achievement and other measures of 
student success, overall and for subgroups of students identified in the statute as high 
priority? 

e What effect does the program have on student and parent perceptions of school safety and 
satisfaction, and on parents’ involvement in education of their children? 

e Does the program change students’ instructional environments and opportunities? 


Design: 

The evaluation will primarily compare outcomes of approximately 1,800 student applicants 
randomly assigned by lottery to either receive a scholarship or not receive a scholarship. 
Lotteries of program applicants were conducted in spring 2012 (cohort 1), spring 2013 (cohort 
2), and spring 2014 (cohort 3). Data were collected for three follow-up years for each of the 
cohorts and, for students in both the scholarship and nonscholarship groups. The contractor 
administered academic assessments, and conducted student, parent, and principal surveys 
each spring (Spring 2013-16). Prior to the first impact report, descriptive reports based on 
application forms parents filled out when they applied to the OSP, principal surveys, and extant 
data were released. 
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Estimated or Actual Completion Date: 
The second and final impact reports are planned for early spring 2018 and late fall 2018. 


Link to Additional Information: htip://ies.ed.gov/ncee/projects/evaluation/choice soar.asp. 


Parent Information and School Choice Evaluation 


Study Purpose: 

School choice is among the most visible K-12 education policy trends to emerge over the last 
two decades. For school choice to be effective, it seems critical that parents are able to navigate 
school choice systems and process large amounts of complex information about schools and 
application procedures to make informed choices. However, few studies have rigorously 
examined school choice information strategies or attempted to identify effective methods of 
information presentation. This evaluation is designed to address this significant gap in the 
literature. 


Key Question(s) Addressed: 

e Which formats make school choice information displays easiest to understand and use? 
(e.g., Is it better to show school performance data with numbers, graphs, or icons?) 

e How does the amount of information displayed affect understanding and use? (e.g., How 
much information about school performance should be shown? Should displays include 
district averages or parent survey data?) 

e How should school choice information displays be organized? (e.g., How should schools be 
ordered, when a list of options is shown?) 


Design: 

A low-cost, quick turn-around “lab” based experiment was carried out where about 3,500 low- 
income parents of school-aged children participated online. Parents were randomly assigned to 
view one of 72 versions of a school choice information display and then answered survey 
questions about their understanding of the information, ability to use the information, and which 
schools they would select based on the information they were provided. Responses to these 
survey questions will be compared across the strategies tested (e.g., data shown with numbers, 
graphs, or icons) to determine which ones are most promising. 


Estimated or Actual Completion Date: 

A short user-friendly guide, based on the evaluation’s findings, that will help school districts and 
other providers of school choice information design or refine their own parent information 
materials is expected in spring 2018. 


Link to Additional Information: hitp://ies.ed.gov/ncee/projects/evaluation/choice_parent.asp. 


Students with Disabilities 
National Longitudinal Transition Study (NLTS) 2012 


Study Purpose: 

Supporting youth’s successful transition to adult life is a priority under the Individuals with 
Disabilities Education Act (IDEA). IDEA provides federal funds to states and through them to 
school districts to assist in providing special education and related services. States and school 
districts must make a FAPE available to all eligible children with disabilities. FAPE includes the 
provision of special education and related services provided in conformity with an Individualized 
Education Program. IDEA requires that the first IEP to be in effect when a child turns 16 include 
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postsecondary goals and transition services needed to help youth with disabilities complete high 
school prepared to achieve appropriate post-school outcomes, such as postsecondary 
education, jobs, and independent living. The National Longitudinal Transition Study 2012 (NLTS 
2012) is the third in a series examining the characteristics, school experiences, and post-high 
school outcomes of a nationally representative sample of youth with disabilities. NLTS 2012 
focuses on youth ages 13 to 21 (in December 2011), but also includes a small sample of 
students without disabilities to enable direct comparisons of students with and without an IEP. 
This study is part of the congressionally mandated National Assessment of the IDEA and is 
supported with funds authorized under Section 664 of IDEA. 


Key Question(s) Addressed: 

e How do the personal, family, and school characteristics and experiences of youth with 
disabilities differ from those of youth not served under IDEA? (Volume 1; March 2017) 

e How do the characteristics and experiences of youth vary across disability groups? (Volume 
2; March 2017) 

e How have the characteristics and experiences of youth with disabilities changed over time? 

e TO what extent do youth with disabilities make progress through high school compared to 
other youth? 

e Are youth with disabilities achieving the post-high school outcomes envisioned by IDEA, and 
how do their college, training, and employment rates compare to those of other youth? 

e How do these high school and postsecondary outcomes vary with student characteristics? 


Design: 

This descriptive study includes a nationally representative sample drawn from 432 school 
districts and special schools randomly sampled in 2011 and students randomly sampled within 
those districts. Survey data were collected in 2012-2013 from approximately 12,000 in-school 
youth and their parents, of which about 10,000 are students with IEPs representing each of the 
federal disability categories. The surveys asked about the background characteristics of 
secondary school youth and the schools they attend, their health, functional abilities, and 
engagement in school, the academic supports they receive, and their expectations for and steps 
to achieve transitions beyond high school. High school records and postsecondary enrollment 
information from the National Student Clearinghouse are being collected (from 2017-2019) to 
follow the sample of students through high school and beyond. This administrative data will be 
linked with the 2012-2013 survey data to examine high school course-taking and completion, 
and youth’s experiences with college, training, and employment. 


Estimated or Actual Completion Date: 

Two volumes of a report describing the survey results were released in March 2017. The third 
volume of this report, examining trends over time by comparing the NLTS 2012 survey results 
with those from two earlier NLTS surveys, is expected in 2018. 


Key Findings (from the March 2017 volumes): 

e Youth with an IEP are more likely than their peers to be socioeconomically disadvantaged. 
Youth with an IEP are 12 percentage points more likely to live in low-income households 
and are less likely to have parents who are employed or have a college education. Among 
disability groups, youth with intellectual disability and youth with emotional disturbance are 
more socioeconomically disadvantaged and more likely to attend a lower-performing school 
than youth with an IEP overall. 

e The vast majority of youth with and without an IEP feel positive about school, but those with 
an IEP experience bullying and are suspended at higher rates. Like their peers, more than 
80 percent of youth in special education report that they are happy with school and with 
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school staff. However, not only do youth with an IEP more commonly experience some 
types of bullying (e.g., being teased or called names) but, according to parent reports, they 
are more than twice as likely to be suspended or expelled from school. 

e Youth with an IEP are more likely than other youth to struggle academically, yet less likely to 
receive some forms of school-based support. Half of all youth with an IEP report they have 
trouble with their classes, about 15 percentage points more than reported by their peers. 
However, they are less likely to report receiving school-based academic help before or after 
regular hours, although their parents more commonly help with homework and attend a 
parent-teacher conference. 

e Youth with an IEP lag their peers in planning and taking steps to obtain postsecondary 
education and jobs. Substantially fewer youth with an IEP expect to enroll in postsecondary 
education or training, compared to youth without an IEP. Reflecting these gaps, youth in 
special education are almost half as likely as their peers to report taking college entrance 
and placement tests. Forty percent report having recent paid work experience while in high 
school, compared with 50 percent of youth without an IEP. 

e Youth with autism, deaf-blindness, intellectual disability, multiple disabilities, and orthopedic 
impairments are most at-risk for not transitioning successfully beyond high school. Youth in 
these groups are less likely than all youth with an IEP to have key characteristics and 
experiences linked to success after high school, such as performing typical daily living tasks, 
engaging with friends and in school activities, or preparing for college, careers, and 
independent living. 


Link to Additional Information: 
https://ies.ed.gov/ncee/projects/evaluation/disabilities nlts2012.asp. 


Preschool Special Education Programs and Practices Supporting Children with 
Disabilities 

Study Purpose: 

Most recently reauthorized in 2004, IDEA provides funding to states to support special 
education and related services for children and youth with disabilities, including young children 
ages 3-5 years old. Currently there is limited information available on the curricula and 
interventions being used across states to support young children with disabilities. Phase | of the 
Evaluation of Preschool Special Education Practices is being conducted to address the primary 
objective of assessing the feasibility of conducting a large-scale impact study of curricula or 
interventions promoting the literacy, language, and/or social-emotional skills of preschool-age 
children with disabilities. A secondary objective of the Phase | study is to provide nationally 
representative descriptive information about preschool special education programs and the 
specific curricula or interventions being delivered to preschool children with disabilities. 


Key Question(s) Addressed: 
e Which curricula and interventions are used nationally for preschool children with 
disabilities to promote learning of language, literacy, and social emotional skills? 
e How are states and school districts structuring programs to serve children ages 3 
through 5 with disabilities? 
e What staff resources are available to support the instruction of children ages 3 through 5 
with disabilities? 
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Design: 

The study collected survey data from state and district grantees of IDEA funds to obtain 
nationally representative information on the programs, services, curricula and interventions 
available to children ages 3—5 identified for special education services. The survey sample 
included a nationally representative sample of 1,200 school district preschool special education 
coordinators and state Section 619 coordinators in all 50 states and DC. If feasible to conduct, 
preparations for the impact study will occur under Phase I. 


Estimated or Actual Completion Date: 
A descriptive report on the survey findings is expected in fall 2018. 


Link to Additional Information: 
https://ies.ed.gov/ncee/projects/evaluation/disabilities preschool.asp. 


National Evaluation of the IDEA Technical Assistance and Dissemination 
Program: Final Report 


Study Purpose: 

IDEA, which was most recently reauthorized in 2004, provides funds to assist states and local 
educational agencies in making a free appropriate public education available to all eligible 
children with disabilities. Funded at $12.6 billion in FY 2010, IDEA supports early intervention 
services for infants and toddlers with disabilities (birth through age 2) and special education and 
related services for children with disabilities (ages 3 through 21). IDEA permits local educational 
agencies to use a portion of their IDEA funds to develop and implement coordinated early 
intervening services (CEIS) for students who are not currently identified as needing special 
education or related services, but who need additional academic and behavioral support to 
succeed in a general education environment. The use of IDEA funds for CEIS is voluntary, 
except when an LEA is identified with significant disproportionality. IES is conducting studies 
under Section 664 of IDEA to assess the implementation and effectiveness of key programs and 
services supported under the law. 


As specified in IDEA Part D, the Technical Assistance and Dissemination (TA&D) program is to 
provide technical assistance, support model demonstration projects, disseminate useful 
information, and implement activities that are supported by scientifically based research to meet 
the needs of children with disabilities. The National Evaluation of the IDEA TA&D program is 
designed to describe the products and services provided by the TA&D program grantees, state 
and local needs for technical assistance, and the role that the TA&D program plays in meeting 
these needs and supporting implementation of IDEA. The State Deaf-Blind Projects are part of 
the TA&D Program and are the focus of the evaluation’s final report. 


Key Question(s) Addressed: 

e What technical assistance and dissemination activities do State Deaf-Blind Projects provide 
and how does this vary across the states? 

e How do State Deaf-Blind Projects collaborate with other organizations in their state, with 
other technical assistance providers, and across the network of State Deaf-Blind Projects? 

e What are the needs for technical assistance among direct service providers who work with 
children and youth with deaf-blindness? 

e How satisfied are direct service providers with services received from the State Deaf-Blind 
Projects? 
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Design: 

Data collection for the interim report included administering surveys to the national and regional 
level TA&D program grantees, all state IDEA Part B and Part C administrators, and a sample of 
state-level special education program staff. Data were collected between November 2012 and 
March 2013. State-level administrators and staff reported on their receipt of technical assistance 
from TA&D program grantees during the 2010—11 school year. Grantees reported on their goals 
and activities from the beginning of their current grant through the interview date. The funding 
period for the centers included in the interim report varied, with the earliest end date in 2012 and 
the latest in 2014. Data collection for the final report included administering surveys to the 48 
state level deaf-blind projects awarded grants in 2013 and to a sample of local level special and 
general education providers currently working with deaf-blind children. 


Estimated or Actual Completion Date: 
The final report is expected in early 2018. 


Key Findings from the Interim Report: 

e TA&D centers most commonly reported providing technical assistance on the topics of 
"parent and family involvement” and “data systems and use of data for improvement.” States 
identified (a) “General Supervision/Monitoring,” (b) "early childhood transition,” (c) "financing 
of services/financing for special education” and (d) "Response to Intervention” as the topics 
for which they had the greatest need for technical assistance in 2010-11. 

e Many TA&D centers provide technical assistance on similar topics. For example, 14 states 
received "high intensity" technical assistance (i.e., frequent training or consultation) on the 
same topic from 5 different centers. 

e State staff rated the majority of technical assistance experiences they had with TA&D 
centers as "very satisfactory" (71 percent). On average, customers receiving high intensity 
technical assistance were significantly more satisfied than those receiving lower intensity 
(i.e., infrequent training and consultation or web-only support). Satisfaction did vary to some 
degree depending on the special education topic being addressed. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/disabilities idea2004.asp. 


Study of Early Intervention and Special Education Services and Personnel 


Study Purpose: 

IDEA, which was most recently reauthorized in 2004, provides federal funds to states and 
through them to school districts to assist in making a free appropriate public education available 
to all ell eligible children with disabilities. Funded at $12.5 billion in FY 2015, IDEA supports 
early intervention services for infants and toddlers with disabilities (from birth through age 2) 
(IDEA Part C) and special education and related services to children with disabilities (ages 3 
through 21) (IDEA Part B). IDEA permits LEAs to use a portion of their IDEA funds to develop 
and implement coordinated early intervening services for students not currently identified as 
needing special education or related services but who need additional academic and behavioral 
support to succeed in a general education environment. The use of IDEA funds for CEIS is 
voluntary on the part of the LEA, except when an LEA is identified with significant 
disproportionality. Under Section 664 of IDEA, IES is conducting studies to assess the 
implementation and effectiveness of key programs and services supported under the law. This 
study is supporting the analysis of extant data to examine early intervention and special 
education service delivery and the personnel providing services. 
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Key Question(s) Addressed: 

e How has the delivery of early intervention services changed over time, and how do these 
services differ for subgroups defined by age, race/ethnicity, and state? 

e How have special education services changed over time, and how do these services differ 
for subgroups defined by age, race/ethnicity, disability category, and state? 

e How has the distribution of personnel providing special education services changed over 
time? 


Design: 

This study includes new descriptive analysis of extant data available in public use or restricted 
formats. Among the data sources that are being used are cross-sectional Section 618 data 
submitted by states to the Department and the NCES Schools and Staffing Survey. 


Estimated or Actual Completion Date: 
The report is expected in early 2018. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/disabilities persserv.asp. 


Impact Evaluation of Training in Multi-Tiered Systems of Support for Behavior 
(MTSS-B) 


Study Purpose: 

Training school staff in supporting the behavior of all students is becoming increasingly 
attractive to districts and schools as a vehicle for school improvement. Implementation of multi- 
tiered systems of support for behavior (MTSS-B) is an approach to improving school and 
classroom climate as well as student outcomes. MTSS-B is a multi-tiered, systematic framework 
for teaching and reinforcing behavior for all students as well as for providing additional support 
to those who need it. Over a third of U.S. districts report implementing multi-tiered systems of 
behavior support at the elementary school level. Recent small-scale studies have shown the 
promise of MTSS-B. This evaluation occurs under the National Assessment of the /ndividuals 
with Disabilities Education Act (IDEA), which permits school districts to use a portion of their 
IDEA funds to provide coordinated early intervening services to students who are not currently 
identified as needing special education or related services, but who need additional academic 
and behavioral support to succeed in a general education environment, such as MTSS-B. 


Key Question(s) Addressed: 

e What MTSS-B training and support activities were provided? What MTSS-B activities 
occurred in the schools receiving MTSS-B Training? How do these MTSS-B activities differ 
from those in schools that do not receive the training? 

e What is the impact on school staff practices, school climate and student outcomes of 
providing training in the MTSS-B framework plus universal (Tier |) positive behavior supports 
and a targeted (Tier II) intervention? 

e What are the impacts for relevant subgroups (e.g., at-risk students)? 


Design: 

This is a randomized trial of the impact of training in MTSS-B on school climate, school staff 
practice, and student outcomes. The contractor, with assistance and input from the Department 
and in consultation with a panel of experts, competitively selected a MTSS-B training provider, 
the Center for Social Behavior Support, which is a collaboration between the Illinois-Midwest 
Positive Behavior Interventions and Supports network at the School Association for Special 
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Education in DuPage, Illinois and the Positive Behavior Interventions and Supports network 
Regional Training and Technical Assistance Center at Sheppard Pratt, in Maryland. 
Approximately 90 elementary schools were randomly assigned to either training in MTSS-B 
including universal supports (Tier l) plus targeted interventions for at-risk students (Tier II) or a 
business-as-usual control group. Treatment schools are receiving training in MTSS-B prior to 
and across two school-years, 2015-2016 (Tier 1) and 2016-2017 (Tiers | and II), and implement 
MTSS-B across these two years. Data collection will include a staff survey, teacher ratings of 
student behavior, classroom observations, site visits, and student records data. 


Estimated or Actual Completion Date: 
The report is expected in 2019. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/disabilities MTSSB.asp. 


Pathways to Career or College 
Evaluation of the Pell Grant Experiments Under the Experimental Sites Initiative 


Study Purpose: 

Federal Pell grants are considered the foundation of higher education financial aid for low- 
income students. However, under current rules, otherwise income-eligible students who already 
have a bachelor’s degree (BA) or who want to enroll in short-term (less than 15 weeks and 600 
hours) programs are restricted from obtaining these grants. Given unemployment rates above 
8.5 percent in 2011, and reports of unfilled openings for skilled jobs in some occupations, 
postsecondary institutions called for expanding Pell grants to help fill the skill training gap for 
low-income workers. In response, the Office of Federal Student Aid, under the Experimental 
Sites Initiative authorized by section 487A(b) of the Higher Education Act of 1965, is conducting 
demonstrations to test the impacts of eliminating the BA restriction (Experiment #1) and 
significantly lowering the minimum clock hours/duration restriction (Experiment #2) for students 
interested in vocational training in high-demand fields. The IES designed and is overseeing a 
rigorous evaluation of these experiments. 


Key Question(s) Addressed: 

e Does expanding Pell grant eligibility to include income-eligible students with a bachelor's 
degree and/or to cover shorter-term programs improve access to job training? 

e Does expanding Pell grant eligibility to these groups affect financial aid receipt and/or 
student debt? 

e Do these two Pell grant experiments improve persistence and completion rates? 

e Is there any evidence of an impact on employment and earnings? 


Design: 

Close to 50 higher education institutions that chose to participate have identified nearly 3,000 
students eligible for the experiments between the 2012-2013 and 2016-2017 financial aid award 
years. Students were randomly assigned to receive a Pell grant or not to receive a Pell grant in 
their financial aid package. Student administrative data from participating higher education 
institutions will be collected in the fall of 2017 and 2018 to allow for the maximum number of 
study participants to complete their programs. Data collection will also include student 
administrative records on financial aid receipt from the Office of Federal Student Aid and, 
potentially, earnings data from the Social Security Administration. 
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Estimated or Actual Completion Date: 
The report for the study is expected in fall 2019. 


Link to Additional Information: 
https://ies.ed.gov/ncee/projects/evaluation/pathways _pell.asp. 


Enhancing Advising to Improve College Fit in Upward Bound 


Study Purpose: 

Growing concern over college enrollment and completion rates has heightened interest in 
strategies to improve the outcomes of low-income students, including those in the Department’s 
high-profile college access program Upward Bound. One aspect of this concern is that many 
low-income high school students fail to enroll in colleges that are well aligned with their 
qualifications, talents, and needs. Research suggests that this kind of “undermatching” or lack of 
college fit can have longer-term consequences for completion and later earnings. This study 
evaluates the effectiveness of a low-cost enhancement to current college advising approaches 
in Upward Bound that is designed to improve college fit and persistence. The approach includes 
professional development for Upward Bound advisors and user-friendly packets of information 
for students that demonstrate the value of considering a range of institutional indicators in their 
college application, search, and acceptance process. 


Key Question(s) Addressed: 

e Canan enhanced college advising approach improve upon what Upward Bound grantees 
are already doing, i.e., does it have positive effects on the number of colleges to which 
Upward Bound participants apply, the quality/selectivity of the colleges in which they enroll, 
and their persistence? 

e In what types of grantees is this approach most effective and with what types of students? 


Design: 

About 200 Upward Bound grantees that volunteered were randomly assigned in spring 2015 so 
that half received the professional development and packets to begin using with their rising 
2015-16 seniors (the treatment group) and half did not receive the enhancements during the 
study period (the control group). A survey administered in spring 2016 to seniors in both sets of 
projects collected information about their college planning, including how many and to which 
colleges they applied. The study will also examine those seniors’ later enrollment and 
persistence in college using administrative records. 


Estimated or Actual Completion Date: 

The first report will examine the impacts of the enhanced advising on early indicators of college 
going (number of college applications, selectivity of the colleges to which students applied, 
FAFSA completion by March 15) and is expected to be released in spring 2018. A second report 
examining impacts on college enrollment and fit is expected in fall 2018. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/pathways upward.asp. 
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The Effectiveness of Text Messaging to Support College Transition of GEAR UP 
Students 


Study Purpose: 

The Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) is one 
of the Department’s college access programs, funding states or local partnerships of districts 
and postsecondary institutions to serve students in high need schools beginning in 7th grade. 
The 2008 amendments to the Higher Education Act allowed GEAR UP grantees to serve 
participating students beyond high school graduation and into a first year of college, when they 
might be dispersing to a variety of college campuses. This demonstration evaluates a low-cost 
way to provide these services, building on emerging evidence that customized text messages 
can help students overcome logistical and behavioral challenges that might otherwise derail 
their college matriculation and persistence into sophomore year. The messages include 
reminders and information relating to college registration, course selection, financial aid award 
and renewal, meeting with college advisors and faculty, and tuition payments, and provide a 
way to access real-time support from GEAR Up counselors. 


Key Question(s) Addressed: 

e Do the text-based reminder strategies improve GEAR UP students’ rates of college 
enrollment and persistence? 

e For which types of students is the strategy more or less beneficial? 


Design: 

Across about 80 GEAR UP high schools, approximately 6,000 seniors in school years 2015-16 
or 2016-17 were randomly assigned to receive either the college transition services grantees 
originally proposed in their applications or those regular transition services plus the customized 
reminders and support through text messages. Reminders and support began at the end of 
students’ high school senior year and continue into the spring of their expected first year of 
college. The study team administered a survey before students were randomly assigned in 
order to collect information on students’ experiences with college advising and their intended 
college (so that the messages can be tailored to individual schools’ deadlines and 
requirements). College enrollment and persistence, as well as FAFSA renewal, will be 
measured using administrative records. 


Estimated or Actual Completion Date: 
The first report examining impacts on initial college enrollment is expected in early 2019. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/pathways gearup.asp. 


Other 


Implementation Study of State Supports under Title | for Reducing School 
Dropouts 


Study Purpose: 

Title 1, Part A of the ESSA requires states to support local educational agencies in providing 
effective transitions of students at all levels of schooling, especially middle grades and high 
school, to decrease the risk of students dropping out (Section 1111(g)(1)(D)). Title IX, Section 
9208 of ESSA requires an evaluation of these efforts. This report will describe the 
implementation of dropout prevention strategies at the state and district levels and report recent 
dropout rate trends. 
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Key Question(s) Addressed: 

e How do states and districts identify students at risk of dropping out? 

e What strategies do districts use to help students transition from elementary to middle school 
and from middle to high school? What services or options do districts offer to students at risk 
of dropping out? How do states support these efforts? 

e What are recent trends in dropout rates at the national and state levels? 


Design: 

Data will be collected from all 50 states and the District of Columbia, and a nationally 
representative sample of school districts through surveys during the 2017-18 school year to 
inform the first two research questions. Extant data in dropout rates will also be collected to 
inform the third research question. 


Estimated or Actual Completion Date: 
An evaluation brief describing findings is expected in summer 2019. 


Evaluation of Investing in Innovation 


Study Purpose: 

The Investing in Innovation (i3) Fund was established by the American Recovery and 
Reinvestment Act of 2009 to provide funds to local educational agencies and nonprofits with a 
record of improving student achievement. The program awards competitive grants to implement 
and evaluate educational interventions to improve student academic outcomes. 


The i3 program awarded three different tiers of grants. The amount of funding awarded was 
aligned with the strength of the prior evidence supporting the proposed intervention as well as 
the proposed implementation scale. The smallest, or “Development,” grants supported 
developing and testing interventions with limited or no prior evidence. Interventions with 
moderate evidence of effectiveness could receive a larger “Validation” grant, to implement and 
test the intervention in a broader population or in new contexts. The largest, or “Scale-up,” 
grants supported interventions with strong prior evidence of effectiveness to be implemented 
and tested on a much larger scale. 

The 13 program funded seven cohorts of grantees from 2010 through 2016. The program has 
awarded more than $1.4 billion to 172 grantees across the three tiers. One of the i3 program's 
key priorities is evaluating the impact of the funded interventions. Each grantee was required to 
fund an independent evaluation using local evaluators. To maximize what could be learned from 
these evaluations, the Department is conducting an Evaluation of i3, which (1) provides 
comprehensive technical assistance to local evaluators for all grantees, and (2) summarizes 
both the quality of and findings from the evaluations. 


Key Question(s) Addressed: 

e Were the 13 evaluations strong? 

e Did the i3 evaluations find the interventions to be implemented with adequate fidelity? 

e Did the i3 evaluations find that the interventions improved student academic outcomes? 


Design: 

This study is descriptive and includes technical assistance to support the i3 local evaluators. In 
addition to providing technical assistance, the contractor will characterize whether or not the 
evaluations meet several criteria used to define a strong study and provide a report for the 
program office each year about the number of grantees meeting the i3 evaluation-related 
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Government Performance and Results Act measures. The contractor will also summarize the 
results of the i3 evaluations. 


Estimated or Actual Completion Date: 
The first report, which will focus on findings for the 67 i3 evaluations completed by May 2017, is 
expected in spring 2018. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/assistance_ita.asp. 


Progress and Challenges in Developing Tiered Quality Rating and Improvement 
Systems (TQRIS) in the Round 1 Race to the Top-Early Learning Challenge (ELC) 
States 


Study Purpose: 

The Race to the Top—Early Learning Challenge Program (RTT-ELC) aims to improve early 
learning and development programs so children enter kindergarten ready to succeed. The 
program awarded $1 billion in four-year grants to 20 states in three rounds between 2011 and 
2013. The program promotes reform in five key areas: (1) state systems; (2) high-quality, 
accountable programs; (3) early learning and development outcomes for children; (4) workforce; 
and (5) measurement. The second area focuses on the design and implementation of TQRIS 
that can provide parents and other stakeholders with information on the quality of early learning 
programs. To better understand the TQRIS that RTT-ELC grantees developed, this descriptive 
study will: (1) examine the structure and characteristics of grantee states' TQRIS, (2) analyze 
state administrative data to examine patterns in the participation and distribution of programs in 
TQRIS, and (3) synthesize findings from grantee states with completed TQRIS validation 
studies. This study focuses on the nine Round 1 states, which received over $500 million in 
2011. 


Key Question(s) Addressed: 

e How are TQRIS structured and implemented in the nine Round 1 RTT-ELC states? 

e How are TQRIS ratings defined, collected, and generated in the nine Round 1 RTT-ELC 
states? 

e What are the patterns in TQRIS participation, the distribution of programs and children 
across the rating levels and the movement of programs and children across the rating levels 
since 2011? 

e How do TQRIS characteristics (include structure, policies, and practices) relate to program 
movement up the rating levels and achievement of the highest rating level? 

e What have completed RTT-ELC state validation studies found about the relationship 
between TQRIS tiers and program quality and the relationship between children's 
development and TQRIS ratings? 


Design: 

This descriptive study collected various data from the nine Round 1 RTT-ELC grantee states. In 
fall 2014, the study collected and conducted a targeted review of documents describing the 
structure of TQRIS, including component measures and the quality indicators used to evaluate 
preschool programs, and how these are combined to generate overall ratings. In fall 2014 
through winter 2015, the study also conducted interviews with state administrators to confirm 
and clarify the information obtained from documents and gather information that could not be 
obtained from the document reviews. 
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To address the third and fourth research questions above, the study collected administrative 
data from the Round 1 RTT-ELC grantee states. The study will answer the last question by 
conducting a systematic review of up to nine completed RTT-ELC grantee state validation 
studies available in 2017. 


Estimated or Actual Completion Date: 
A report addressing the first two questions is expected in fall 2017. Evaluation briefs to address 
the third, fourth, and fifth questions are expected in fall 2018. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/other_rtt.asp. 


Evaluation of the Comprehensive Technical Assistance Centers, FY 2012 
Grantees 


Study Purpose: 

The Comprehensive Technical Assistance Centers is a federally funded program currently 
authorized under the Educational Technical Assistance Act of 2002. The Department awarded 
five-year grants in FY 2012 to 22 Comprehensive Technical Assistance Centers with the 
purpose to help state education agencies build their capacity to implement state-level initiatives 
and to support district- and school-level initiatives that improve educational outcomes for all 
students, close achievement gaps, and improve the quality of instruction. The FY 2014 
appropriation for the Centers was $48.4 million. This study will inform the Department of 
Education, the Comprehensive Center program, and the larger field about the design, 
implementation, and outcomes of the Centers’ work. 


Key Question(s) Addressed: 

e Program Design: How did the Centers design their work? How did Centers define capacity 
building? Did their definitions change over time? If so, how? What theories of action did 
Centers use to guide their general capacity-building work? Did the theories change over 
time? If so, how? How did Centers assess the needs of their constituencies? 

e Program Implementation: How did the Centers operate? What strategies did Centers employ 
to achieve their outcomes? To what extent did Centers implement technical assistance to 
their constituencies as planned? To what extent and how did Centers collaborate with each 
other? 

e Program Outcomes: What was the result of the Centers’ work? To what extent did Centers 
achieve their goals and objectives? 


Design: 

This evaluation is a multiyear descriptive study examining the Centers’ programs. Data on the 
Centers’ activities and outcomes will be collected during the FY 2015, FY 2016, and FY 2017 
program years. The evaluation will describe how the individual Centers intend to build SEA 
capacity (their theories of action) and document what types of activities they actually conduct to 
build capacity. The evaluation plans to focus on Center projects in two priority areas: great 
teachers and leaders and early learning. Data collection will include: (1) the Centers’ 
management plans and technical assistance activity data; (2) interviews with staff from each 
Center; (3) interviews with technical assistance recipients; (4) a survey of Center staff; and (5) a 
survey of technical assistance recipients. This approach will yield a diverse set of data that can 
be analyzed and summarized using qualitative research methods and simple quantitative 
tabulations. Additionally, a small set of project profiles will be put together using the data to 
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illustrate in more depth the particular strategies that Centers may use to support capacity 
building and achieve planned outcomes. 


Estimated or Actual Completion Date: 
A report describing findings is expected in fall 2018. 


Link to Additional Information: 
http://ies.ed.gov/ncee/projects/evaluation/other_techcenters12.asp. 


Impact Evaluation of Parent Messaging Strategies on Student Attendance 


Study Purpose: 

Student attendance is a strong predictor of student success, even in the early school years. In 
some communities, a quarter of all students in kindergarten through third grade are chronically 
absent, and this is most prevalent among low-income students. Under the recently reauthorized 
Elementary and Secondary Education Act (ESSA) many states plan to hold schools 
accountable for reducing chronic absenteeism. One potential low-cost intervention that schools 
are increasingly trying involves text messaging parents to provide relevant tips and motivation to 
improve their child’s attendance. This evaluation is designed to provide evidence on the 
effectiveness of such an intervention for students attending low-performing elementary schools. 
A novel aspect of the intervention is that it is adaptive. Initially, parents will be treated with “light- 
touch” messaging in the fall, and those that do not appear to be responsive will subsequently 
receive more intensive messaging in the spring. 


Key Question(s) Addressed: 

e What is the impact on student attendance of using text messaging to provide parents with 
basic information related to their child’s attendance? Does it matter if messages are 
positively or negatively framed? 

e For parents who do not respond to the low-intensity messaging strategies, which intensified 
strategy works better to improve attendance, one that includes direct outreach from school 
staff or one that uses automated methods to improve motivation and behavioral skills? 

e Do the combinations of fall and spring messaging strategies (i.e., the adaptive interventions) 
have effects on end-of-year attendance and achievement when compared to each other and 
to business-as-usual attendance strategies? 

e How is the messaging intervention implemented, and what are its costs? 


Design: 

The evaluation will use a variant of a typical random assignment design, called a sequential, 
multiple assignment, randomized trial. This design will measure the effectiveness of different 
initial messaging strategies, and examine if there are any benefits to following up with more 
intensive strategies for parents who do not respond initially. Data will be collected to examine 
both the implementation of the intervention and the impact of the intervention on student-level 
outcomes, such as attendance and achievement. Specifically, information gathered from the 
text-messaging vendor's platform, a brief log that school staff will complete to document their 
parent outreach activities, interviews with district staff, and a survey of parents will be used to 
assess how well the intervention was implemented and to examine its costs. Information 
gathered from extant district records, including student absences and test scores, will be used 
to examine the impact of different messaging strategies. 


Estimated or Actual Completion Date: 
A report describing findings is expected in 2019. 
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Link to Additional Information: 
https://ies.ed.gov/ncee/projects/evaluation/other_messaging.asp. 


Policy and Program Studies Service 


Teacher Education Assistance for College and Higher Education Grant (TEACH) 
Study 


Study Purpose: 

The TEACH Grant program provides grants up to $4,000 a year to students who are completing 
coursework needed to begin teaching. If a recipient does not complete four years of service ina 
high-need subject at a high-need school within eight years after completing their coursework, 
their grant funds are converted to a direct unsubsidized loan. This study examines how 
institutions support and inform students who are eligible for TEACH Grants. In addition, the 
study examines why some participants do not meet TEACH Grant service requirements. Lastly, 
the study examines factors associated with grant recipients meeting and not meeting service 
requirements. 


Key Question(s) Addressed: 

e How are TEACH Grants administered in institutions of higher education? 

e Why do some TEACH Grant recipients fail to meet program service requirements? 

e What are the factors associated with TEACH Grant recipients meeting and not meeting the 
grant service requirements? 


Design: 

The study includes surveys of 1) 479 institutions that administered at least 10 TEACH Grants in 
the 2014—15 award year and 2) a sample of 500 TEACH Grant recipients. In addition, the study 
includes case studies of six institutions and an analysis of administrative data. 


Estimated Completion Date: 
The final report is scheduled for completion by the spring of 2018. 


National Survey on High School Strategies Designed to Help At-Risk Students 
Graduate 


Study Purpose: 

This nationally representative survey of high school administrators is examining strategies that 
high schools use to reduce students’ likelihood of dropping out of high school and to increase 
their likelinood of attaining a high school credential. The survey seeks information on the 
prevalence of high school graduation strategies, the students who participate in them, and how 
high schools deliver services or interventions as part of that strategy. The Department will 
release a set of issue briefs based on the survey data to describe the prevalence and 
characteristics of dropout prevention strategies for at-risk youth and will compare high schools 
with high and low graduation rates, among other school characteristics. 


Key Question(s) Addressed: 

e What strategies are high schools implementing to help students stay in school and 
graduate? Do these strategies vary for high schools with high or low graduation rates? 

e How many students are served through each of these strategies? Are the strategies focused 
on particular student populations? 
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e How do schools deliver services or interventions for each of the strategies? What specific 
services are provided, and who provides the services? 


Design: 
The study is conducting a web-based survey of a nationally representative sample of 
approximately 2,000 high school administrators. 


Estimated Completion Date: 
Issue briefs are scheduled for completion between winter 2016-17 and winter 2017-18. 


Implementation Study of the Turnaround School Leaders Program 


Study Purpose: 

This study will examine the implementation of the Turnaround School Leaders Program (TSLP). 
This study seeks to generate information to help policymakers and practitioners who struggle 
with the challenges of developing leaders to turn around low-performing schools and to add to 
the field’s general body of knowledge about developing turnaround leadership pipelines. 


Key Question(s) Addressed: 

e How do TSLP projects identify, develop, and support leaders for low-performing schools? 

e What role do project partners play in implementing projects and helping grantees to achieve 
project goals? 

e How have grantees modified projects to adapt to challenges or meet the demands of 
changing circumstances? 

e How are grantees measuring the success of their TSLP projects, and do early outcome data 
show promising results? 

e How have the turnaround school leaders’ grants contributed to developing a sustainable, 
long-term pipeline of leaders for turnaround schools? 


Design: 

The study will include surveys of 12 TSLP cohort 1 grantees and case studies of seven TSLP 
cohort 1 grantees, including each grantees’ partners; an analysis of extant data, including 
grantee applications, early outcomes data, and other relevant project-specific data. 


Estimated Completion Date: 
The final report is scheduled for completion in spring 2018. 


Study of School Climate Transformation Grants 


Study Purpose: 

The study will describe how states and school districts that participate in the School Climate 
Transformation Grant (SCTG) program are coordinating services and supports with certain 
other related federal programs administered by the Departments of Health and Human Services 
and Justice. 


Key Question(s) Addressed: 

e What does coordination between SCTG and Project AWARE grantees look like? 

e What does coordination between SCTG and U.S. Department of Justice grantees look like? 
e What did grantees say about the value of coordination? 

e What were the challenges and lessons learned? 


FY 2017 Annual Performance Report and FY 2019 Annual Performance Plan—U.S. Department of Education 


155 


APPENDICES 


Design: 

The study will conduct telephone interviews with representatives from 38 state and local 
grantees to explore the ways in which grantees coordinate services, the benefits that grantees 
have experienced from program coordination, and challenges and lessons learned. 


Estimated Completion Date: 
The report is scheduled for completion in spring 2018. 


Study of Title | Schoolwide and Targeted Assistance Programs 


Study Purpose: 

The study will examine how Title | schoolwide programs use the schoolwide flexibility to design 
services and strategies to address the needs of low-achieving students and subgroups, and 
how such strategies compare to approaches used in targeted assistance programs. The study 
will include interviews and analysis of extant data in approximately 30 Title | schools, including 
both schoolwide and targeted assistance programs, as well as surveys of principals and district 
administrators for a representative sample of Title | schools. 


Key Question(s) Addressed: 

e How do schoolwide and targeted assistance programs use Title | funds to improve student 
achievement, particularly for low-achieving subgroups? 

e How do districts and schools make decisions about how to use Title | funds in schoolwide 
programs and targeted assistance programs? 

e To what extent do schoolwide programs commingle Title | funds with other funds or 
coordinate the use of Title | funds with other funds? 


Design: 

The study will include both in-depth case studies and surveys of a nationally representative 
sample of 350 districts and 1,400 schools. The case studies will include approximately 35 Title | 
schools including both schoolwide and targeted assistance programs. Data collection for the 
case studies will include site visits and interviews with two to four staff members in each school 
and approximately two staff members in each district, as well as extant documents and data, 
including Title | budgets and plans. 


Estimated Completion Date: 
The report is scheduled for completion in spring 2018. 


Study of Digital Learning Resources for Instructing English Learners 


Study Purpose: 

This study will examine the use of digital learning resources (DLRs) to support the English 
language acquisition and academic achievement of English learners in K-12 education. The 
study will explore the range of DLRs that are available for use with English learner students, 
examine how districts and schools select and use these apps, and consult an expert panel of 
technology developers, practitioners, and education researchers for ideas on ways to improve 
the design and use of apps to support learning for English learner students. The study will 
culminate in a final report that presents findings from the study, as well as two short field- 
focused toolkits or guides for educators and technology developers that present key information 
from the study in a manner that will be accessible and useful for those audiences. 
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Key Question(s) Addressed: 

e How do districts and teachers identify and select DLRs in general? How do districts and 
teachers identify and select DLRs specifically to support English learner students? 

e What are the types and characteristics of DLRs that districts most commonly report as used 
to support English learners? What are the types and characteristics of DLRs that teachers 
most commonly report that they use in instructing and structuring learning activities for their 
English learner students? 

e How do teachers of English learner students use DLRs in the instruction of English learner 
students? 

e To what extent do teachers receive professional development or other supports for effective 
use of DLRs for instruction? Which professional development approaches do teachers 
report to be most helpful in supporting their use of DLRs in instruction? 

e What are barriers to and supports for (1) the use of DLRs in instruction of English learner 
students and (2) the use of DLRs by students at home? How can districts, schools, and DLR 
developers address these? 

e How do districts and teachers define and measure the success of their use of technology to 
support English learner students? 

e How could developers and practitioners improve the usefulness of DLRs for instructing 
English learner students? 


Design: 

The study will explore the range of such apps that are available for use with English learner 
students, examine how districts and schools select and use these apps, and consult an expert 
panel of technology developers, practitioners, and education researchers for ideas on ways to 
improve the design and use of apps to support learning for English learner students. It will rely 
on six key components: (1) a literature review; (2) market research on existing DLRs for K-12 
instruction; (3) survey of school districts; (4) survey of teachers; (5) case studies; and (6) an 
expert panel of technology developers, practitioners, and education researchers. 


Estimated Completion Date: 
The report is scheduled for completion in spring 2018. 


Study of the Title Ill Native American and Alaska Native Children in Schools 
Program 


Study Purpose: 

The study will examine services and strategies for English learners in the Native American, 
Alaska Native, Native Hawaiian, and Pacific Islander communities that are supported through 
the Native American and Alaska Native Children in Schools (NAM) program. The study includes 
case study of 25 NAM grantees, and examines: (1) the types of services and strategies that 
NAM funds support to address the instructional needs of these communities and develop 
student proficiency in both English and (optionally) native languages; (2) how grantees plan, 
implement, and evaluate their respective projects; (3) how grantees coordinate and prioritize the 
use of NAM funds in relation to other federal, state, and local resources; and (4) how grantees 
use data and evidence to inform program implementation and meet the Department’s reporting 
requirements. 


Key Question(s) Addressed: 


e How does the NAM program support grantees in providing services to Native American, 
Alaska Native, Native Hawaiian, and Pacific Islander children, especially through teacher 
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training and professional development, curriculum development, parent engagement, and 
English (especially academic English) and native language instruction? How do NAM 
grantees incorporate technology to support and/or preserve heritage languages through 
dual-language or other forms of English language instruction? How many students are 
served through each of these strategies? Are the strategies focused on particular student 
populations? 

e What challenges do NAM grantees face in providing funded services, and what steps have 
grantees taken to overcome these challenges? To what extent does the Department or 
other external entities provide support to overcome these challenges? What lessons have 
the grantees learned? 

e How do schools deliver services or interventions for each of the strategies? What specific 
services are provided, and who provides the services? 

e What are the roles and responsibilities among (as applicable) tribal entities, public schools, 
local education districts, and state agencies in implementing NAM grants and meeting 
federal reporting requirements? 

e What are NAM grant stakeholders’ perceptions of community participation and student 
engagement in language instruction and other educational programs? 


Design: 

The study will include site visits to NAM grantees to obtain detailed information regarding how 
they are implementing the NAM program and meeting the needs of their students and 
communities, as well as telephone surveys or interviews of local grant coordinators and state 
directors of Indian education to inform the case study site visits. The study will also utilize extant 
data, including grant applications. 


Estimated Completion Date: 
The report is scheduled for completion in fall 2018. 


Evaluation of the Title |, Part D Neglected or Delinquent Program 


Study Purpose: 

The study will examine the implementation of educational programs for children and youth in 
residential facilities and correctional institutions funded under Title |, Part D of ESEA. The study 
will be informed by surveys of state grantees and local subgrantees to examine the types of 
services and strategies that Part D funds support, how state and local agencies assist students 
in transitioning back to schools, how state correctional facilities implement institution-wide Part 
D projects, and how grantees assess the educational outcomes of participating students. 


Key Question(s) Addressed: 

e What types of services and strategies do Title 1, Part D funds support for children and youth 
in correctional and child welfare settings? 

e How do correctional facilities and child welfare agencies assist students in transitioning back 
to districts and schools, including those outside their jurisdictions? 

e How do state correctional facilities plan and implement institution-wide Part D projects? 

e How do grantees assess the educational outcomes of students participating in 
Part D-funded educational programs? 


Design: 

The study will include a review of extant data, a review of literature related to programs for 
neglected and delinquent youth, surveys of state and local coordinators of Title |-Part D funded 
programs, and site visits to state agencies, school districts, correctional institutions, and child 
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welfare facilities to obtain more detailed information on how grantees and subgrantees are 
implementing the Part D programs and how they are meeting the needs of their students. 


Estimated Completion Date: 
The report is scheduled for completion in spring 2018. 


Evaluation of the Migrant Education Program 


Study Purpose: 

This study will examine how states, districts, and schools are providing instructional supports 
and assessing highly mobile migratory students, as well as examine state plans for 
implementing the new accountability requirements (as they pertain to migratory students) under 
the ESSA. The Migrant Education Program is a state program, giving states flexibility in how 
they allocate funds to serve migratory students, allowing the local entities that serve migratory 
students to be both local educational agencies and local operating agencies. 


Key Question(s) Addressed: 

e How do state and local grantees respond to federal requirements for serving migratory 
children? 

e What services are provided to migratory students? 

e How do Migrant Education Program grantees collaborate with other programs and agencies 
to address the needs of highly mobile students? 

e How do migrant programs support students in earning high school diplomas and 
equivalency, and preparing for postsecondary education and the workforce? 


Design: 

The study will include survey of states, a nationally representative sample of district programs 
serving migrant children, site visits to state and local grantees. The contractor shall prepare a 
final report that integrates findings from the surveys and case studies as well as from an extant 
data analysis. 


Estimated Completion Date: 
The report is scheduled for completion in fall 2018. 


Evaluation of the Indian Education LEA Grants Program 


Study Purpose: 

This study will examine the implementation of the Indian Education Local Education Agency 
(LEA) Grants Program funded under Title VI of the ESEA, as amended by the ESSA. The study 
will document the scope of activities funded by the Indian Education LEA Grants Program and 
will examine the LEA and tribal-level implementation of the grants. Specifically, the study will 
examine the processes used to identify and count eligible children and how grantees establish 
LEA program priorities and implement grant-funded services. 


Key Question(s) Addressed: 

e What strategies do grantees use to identify and count program-eligible children? 

e How do grantees plan services? 

e What services do Indian Education LEA Grants fund? 

e How do grantees assess the project outcomes of students participating in Indian Education 
LEA Grants Program-funded educational programs? 
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Design: 

This study will consist of four Key components: (1) analysis of extant data including APRs and 
Electronic Application System for Indian Education data; (2) review of relevant literature; (3) 
survey of 1,300 coordinators of Indian Education LEA grant programs; and (4) case studies of 
nine districts. 


Estimated Completion Date: 
The report is scheduled for completion in winter 2018-19. 


Study of Weighted Student Funding (WSF) Systems 


Study Purpose: 

This study is examining WSF and school-based budgeting (SBB) systems, which are methods 
for providing funds to schools based on the numbers and, in the case of WSF, types of students 
they served. The study will examine how districts have implemented SBB and WSF systems for 
allocating funds to schools and how these districts and their schools compare with districts 
using traditional systems for allocating school resources. 


Key Question(s) Addressed: 

e How are resources allocated to schools in districts with SBB or WSF systems compared with 
districts with more traditional resource allocation practices? 

e Inwhat ways do schools have autonomy and control over resource allocation decisions, and 
how does this vary between districts with SBB or WSF and other districts? 

e How has the implementation of WSF systems affected the distribution of funding provided to 
schools? 

e What challenges did districts and schools experience in implementing SBB and WSF, and 
how did they respond to those challenges? 


Design: 

To obtain detailed information about the implementation, benefits, and challenges of WSF 
systems, this study will conduct case studies in nine districts using such systems, including site 
visits that include in-person interviews and collection and analysis of extant documents and data 
such as school budgets and planning documents. In addition, the study will administer surveys 
to a nationally representative survey of districts and schools to enable the study to ground the 
case study data in the larger context of WSF, SBB, and traditional methods of resource 
allocation. 


Estimated Completion Date: 
The report is scheduled for completion in winter 2018-19. 
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Appendix F: Glossary of Acronyms and Abbreviations 


Acronym 
AARTS Audit Accountability and Resolution Tracking System 
ACGR Adjusted Cohort Graduation Rate 


AASA American Association of School Administrators 


AP Advanced Placement 


APG 
P 
APR 
CANS 
CAP Goal 


> 
U 


> 
x 


W 


CAR Consolidated Annual Report 
Common Core of Data 


Q 
1) 


Cc 


CDQR Coordinated Data Quality Reviews 
CMO Charter Management Organization 


CoSN 
CROC 
CSAM 


2) 
U 
”n 


2) 
n 


CSP Charter School Program 
CSPR Consolidated State Performance Reports 
CT Career and Technical Education 


i 


OQ 
4 
ep) 


Case Tracking System 
A Digital Analytics Program 


UO 


DLRS 
OMIM 
BOC 
ORT 
OST 


M Case Management System 
U.S. Department of Education 


EDGB 
ELCTA 
ERIC 
ERM 


EDGAR Education Department General Administrative Regulations 


ESEA Elementary and Secondary Education Act of 1965 
ESH Education Superhighway 
ESSA Every Student Succeeds Act 
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EVS Employee Viewpoint Survey 


FAC Federal Audit Clearing House 


GEAR UP 
; . 

: 
H 


/HE == ——_| Higher Education 
TR [Human Resources 


SA 
E 
EA 
R 

Institute of Education Sciences 

National Technical Institute for the Deaf 
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Acronym 
OCFO 
OCIO 


OCTAE 
OELA 
OESE 
OET 
OGC 


QO 
A 


O 
x= 
m8) 


O 


G 


O 


il 
= 


MB 
OPE 
OPEPD 
OPM 
OSEP 
OSERS 
OSP 


ow 


PBCS 
PBIS 
POC 
PPIRS 
PPSS 
PSC 
PTAC 
REAP 
RLIS 
RSA 
RTT 
RTT-ELC 
SAA 
SASS 
SBB 
SCTG 
SEA 
SECOPS 
SFA 
SIG 
SIP 


SLDS 


APPENDICES 


Statewide Longitudinal Data System 


Office of Career, Technical, and Adult Education 


Office of Special Education and Rehabilitative Services 
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B 


To Be Determined 
TEACH Teacher Education Assistance for College and Higher Education Grant 


TIF 


: | 
Pie |TeacherincentveFund SS SSCSCSCSC~* 


What Works Clearinghouse 
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OUR MISSION IS TO PROMOTE STUDENT ACHIEVEMENT AND PREPARATION FOR 
GLOBAL COMPETITIVENESS BY FOSTERING EDUCATIONAL EXCELLENCE AND 
ENSURING EQUAL ACCESS. 


WWW.ED.GOV 


